# Scratch ### Measure regions of mu to improve / not improve Question: This is great. Now I'd like to be able to show the regions of the mu vector that might have been improved, but *don't matter* because of the geometry of sigma^-1. In other words, areas that are less important, because they are less likely to be traded. This could be because: * Variance is too high there so optimizer will discount it * Return is too middle of the pack there, so optimizer will discount it Is this possible? ### Ranks Question: Can we show that improving ranks won't correlate to financials? We've shown that improving MD correlates to financial performance. But can we show that improving the ranks of mu_pred compared to the ranks of mu_true has no guarantee to correlate to financials? To be more clear: say we take mu_pred (the vector) and rank it's values (so each element—asset–now just has it's rank). We do the same for mu_true. We then can compute the element wise rank errors. We can look at the mean rank error, or other similar metrics. Can we show clearly that improving mean rank error (or a similar error) has no guarantee to correlate with financials? See desktop image ### Constraint aware Question: Can we introduce other constraints? So far we've been talking entirely about how Sigma^-1 and the whitened space are what we must focus on becasue that is the geometry the optimizer works in. But now I'd like to consider other constraints that I have that will impact the optimizers geometry. For instance, a key one is that w must sum to 0 (it must be balanced—equal buys and sells). How can I include these types of constraints—the reflect my optimizers geometry—when measuring the error of mu_pred compared to mu_true. ### NLL How would NLL work here? # Open Questions Goal: understand how the optimizer uses the information that's passed to it. We want to know how mu is used by the optimizer. Our optimizers operates in a geometric space—the more we understand about this geometry, the more we understand what errors in mu matter, and how we can create cross val functions that correlate with financials. Concepts that I'm fuzzy on: * return / weight space (primal / dual) * level sets of w * sigma transforming these different spaces (how are these things transformed) * what does it mean to be in a risk adjusted space * How do we undo sigma? R is anisotropic, but we want to get to isotropic where we can apply euclidean distance? * - **Geometry** = the whole induced structure (inner product, norm, angles, contours). Saying “\Sigma^{-1}-geometry” means: _we’re using the geometry that comes from the metric induced by_ \Sigma^{-1}_._ ANSWER: what is the main question on your mind right now? # Personal Clarifications Need to clarify further: * Quadratic form * the map r=\Sigma w is the _structural link_ that makes the two variance forms equivalent, rather than something “derived” by equating the two ellipse equations. * TODO: question for chatgpt—can you say more about, in the following block, *why* the map on weights is not the same? It's adjoint * **1. Returns vs Weights (primal vs dual)** - Returns r live in **asset space**. - Weights w live in the **dual space** (they act as linear functionals: w^\top r). - When you whiten returns by z = \Sigma^{-1/2} r, you are transforming the **primal space**. - The corresponding map on weights is _not_ the same! It’s the adjoint: z^\top = r^\top \Sigma^{-1/2} \quad \implies \quad (w^\top r) = (w_z^\top z), \quad w_z = \Sigma^{1/2} w. - So weights transform with \Sigma^{+1/2}, not \Sigma^{-1/2}." TODO: does the dual space link between R and w have anything to do with covariant and contravariant? # Problem / Solution Outline What concepts do I need to introduce? * Asset space * Returns * Weights * Portfolio variance by direction * Isotropic / Anisotropic * R-geometry * Mahalanobis Distance * Mu * Sigma * MO/MVO * Whitening * W-geometry * * risk-adjusted space * geometry vs space What problem am I trying to solve? How do I want to introduce this? * How is the optimizer "risk aware"? * This is a function of sigma. We use sigma as a constraint. How does that make the optimizer risk aware? * How are inverse sigmas being used here? * What are the different spaces? We have both raw risk space and weight space? * What is the geometric view of our problem? * Why should we think about transforming our space into sigma^-1 space? * How about "middle of the pack" nodes ranked in the middle?