An aggregate with OVER() answers a group-level question without giving up row-level detail — exactly what I need when the output still has to be one row per order, per user, per day.
Learning in public
Notebook
Notes I keep as I learn data science — the ideas that took me a second pass to understand, and the mistakes I would rather not make twice.
Notes
· 8 notesOne lesson per card — open a card for the reasoning behind it
It is the probability of seeing data at least this extreme if the null hypothesis were true — a statement about the data given an assumption, not about the assumption given the data.
Fitting a scaler, imputer, or encoder on the full dataset before splitting lets the test set leak into training through the statistics. Cross-validation looks excellent and production disappoints.
If 1% of transactions are fraud, a model that always predicts no fraud is 99% accurate and completely useless. The metric has to encode which mistake actually hurts.
Facts hold events at a declared grain, dimensions hold the attributes you slice by. That shape is what lets a BI tool generate correct joins without being told how.
One measure returns a different number in every cell, because the rows, columns, slicers, and page filters around it define the context it computes under.
df[mask]['col'] = value is two operations: a selection that may hand back a copy, then an assignment into that copy. The original frame is untouched and pandas raises SettingWithCopyWarning.
Nearly every analysis I have had to redo was not wrong SQL — it answered a question slightly different from the one that was asked. One sentence agreed up front saves a day of work.