P34 – tabular profit prediction without the optimism bias

Hi all. This is the result of a 7-year effort to understand why ML models trained to replace manual/Excel decision processes tend to lose money in production. The core problem is that when you train on a business's historical decisions, the data is biased by which deals were actually taken - so a model fit to it comes out over-optimistic and builds portfolios that look profitable but realize negative yield. We've seen it across Amazon wholesale, retail lending, and other markets inefficient enough that the opportunities look obvious to analysts. P34 is our take on a fix: a pre-trained portfolio-selection model that adjusts how optimistic or cautious to be based on what it observes, instead of inheriting the optimism in historical data. On our synthetic benchmark, naive fitting predicted $170k / realized a negative -$60k; P34 predicted $130k / realized $25k (we’re working on tightening predictions on very difficult markets). It's slow and only fits partially-observed markets and has no value where a full order book exists or in HFT. We’re presenting our first version of benchmark in our github repo that profit-oriented (we call them Profit-as-Regression) models will have to pass/compete in. P34 itself is currently an invite-only API. I’m happy to share more details and will appreciate any feedback on the proposed interface, benchmark and hear if you are solving similar problems in your current project.