Data Science Work

Given a lifter's bodyweight, age, sex, and equipment, how much will they total — and what actually drives strength? A regression problem with a clear, testable answer.

Real IPF competition records from OpenPowerlifting (public domain, via the TidyTuesday project). Real-world messy: dirty categories, impossible outliers (data-entry errors in bodyweight), and missing values (skipped ages, single-lift entries with no total). Cleaning was about half the work: standardizing categories, filtering implausible rows, and deciding when to drop vs impute.

Explore (distributions, correlations, relationships) → engineer features (a strength-to-bodyweight ratio, age bins, one-hot encoding of categories) → model with a proper train/test split: a Linear Regression baseline, then a Random Forest for comparison → evaluate and interpret.

The models predicted a lifter's total to within about 72 kg on data they had never seen, explaining ~71% of the variance. Notably the simple linear model matched the random forest here, a good reminder that on real, noisy data a more complex model does not automatically win. Feature importance quantified what coaches know by feel: bodyweight and sex are the dominant drivers, with equipment and age contributing less.

Across 80,000 real sighting reports: where do they cluster, when do they spike, and can the volume of future reports be forecast? And the question that keeps the project honest, what do those patterns actually measure?

Real reports from NUFORC, the National UFO Reporting Center (via the TidyTuesday project), spanning 1925 to 2014. Messy like any report log: international rows and missing coordinates to filter out, placeholder shapes like na and unknown, and free-text timestamps to parse into a year, month, hour, and weekday.

Geospatial aggregation and a live scatter map of the real coordinates → temporal decomposition across month, hour, weekday, and calendar date → a log-linear trend model fit to the online era with a proper train/test backtest → a transparent likelihood model that scores any month, day, and hour.

Hotspots follow population, not the paranormal: California, Washington, Florida, Texas, and New York lead. The sharpest signal is the clock, reports peak at 9 to 10 pm. The season peaks in July, the week peaks on Saturday, and the Fourth of July is the single busiest date by a wide margin. Report volume grew about 10% a year through the internet era. Every one of those patterns has an ordinary explanation, which is exactly the point: the model describes human reporting behavior, not real events.

Given 22 traits of a mushroom, is it edible or poisonous? A binary classification problem with a brutal twist: one kind of error is annoying, the other is fatal. That asymmetry is the whole point of the study.

The classic UCI Mushroom dataset: 8,124 real mushrooms from a 1981 Audubon Society field guide, every trait categorical. Real cleaning calls too: a constant column that carries no information gets dropped, and 2,480 missing stalk-root values are kept as their own category rather than thrown away. Then every trait is one-hot encoded into 116 features.

Explore (odor alone nearly separates the classes) → one-hot encode → train a logistic regression and a decision tree with a proper train/test split → then judge them the right way: confusion matrix, precision, recall, ROC and AUC, and a tuned decision threshold, not a single accuracy number.

Both models scored above 99.8% on every metric, and made zero false positives. The catch: they still labeled 2 poisonous mushrooms as edible. Every mistake was the dangerous kind, which plain accuracy hid completely. Lowering the decision threshold drives those false negatives to zero. The tree also stayed readable: odor carries most of the signal, and its top rules read like a foraging heuristic. The real lesson transfers straight to fraud detection and disease screening: when one error is far more costly, accuracy is a trap and the confusion matrix is the truth.

Can a neural network learn to read handwriting, and can the trained model then run live in a browser with no server? The problem that put deep learning on the map, taken end to end from raw pixels to a thing visitors can actually play with.

The classic MNIST benchmark: 70,000 real 28 by 28 grayscale images of handwritten digits, split 60,000 for training and 10,000 held back to test on. Pixels normalized to 0 to 1, labels 0 through 9.

Build a convolutional neural network in TensorFlow and Keras: two convolution and pooling blocks to learn visual features, then dense layers to decide → train for 8 epochs and watch the learning curves → evaluate honestly with a confusion matrix → then ship the ~27,000 trained weights into the page and run the network with a from-scratch forward pass in JavaScript, so it predicts your drawing on the spot.

The compact CNN reached 98.7% accuracy on digits it had never seen, and the learning curves stayed healthy (train and validation tracking together, no overfitting). The confusion matrix showed the handful of errors were the humanly reasonable ones, a 4 mistaken for a 9, a 7 for a 1. Best of all, the model is small enough to deploy to the browser: draw a digit on the live pad and the same trained network classifies it instantly, no round trip to any server.

Take 30 years of UFC fights and do what a data analyst is actually paid to do: model the data into a clean relational database, then answer real questions in SQL, from simple joins up to window functions. And make it something a recruiter can run themselves.

A real scrape of ufcstats.com: every event, fighter, and bout. It arrives flat and denormalized, so the first job was modeling it into four linked tables with proper keys: events (789), fighters (4,623), bouts (8,885), and per-fight fight_stats (17,726), spanning 1994 to 2026.

Normalize the schema → write eight analyst queries, each matched to the right technique: JOIN + GROUP BY for win counts, CTE + CASE for finish rates, conditional aggregation for method mix, a self-join for reach, date bucketing for growth, and window functions (gaps-and-islands) for win streaks → then ship the whole SQLite database into the browser with sql.js, so every result is computed live and visitors can run their own SQL.

Jim Miller holds the most wins (28); the window-function query surfaces the legendary streaks, Jon Jones at 19 and Anderson Silva at 16; heavyweights finish 51% by KO while lighter classes go to decision far more often; and a reach advantage, contrary to the myth, wins only about 52% of the time. The best part is the live console: anyone can write their own query against the real database and get an answer instantly, no server involved.

Keyword search breaks when people describe the same thing with different words. Can we search 80,000 real UFO reports by meaning instead, so "a silent triangle over the highway" finds a report about "a triangular craft hovering above the road," even with zero shared words?

The full NUFORC report archive (via TidyTuesday): about 80,000 free-text sighting descriptions. The page pulls the real reports at runtime and indexes a live sample of them.

This is modern NLP: a sentence-embedding transformer (all-MiniLM-L6-v2) is downloaded and run entirely in the browser with WebAssembly (transformers.js), no server. Each report becomes a 384-dimensional vector that captures its meaning; your query becomes one too; and results are ranked by cosine similarity. The index is built once and cached in the browser (IndexedDB), so later searches are instant.

Real semantic search: type a vibe and the model surfaces sightings that mean the same thing, not ones that happen to share a word. It is the same technique behind modern search, recommendations, and retrieval-augmented AI, running live on a static page with no backend at all. Try a query like "lights that changed color then vanished" and watch it rank the closest real reports.

Sumerian is one of the oldest written languages on earth, and it is dead: no native speakers, just clay tablets. Can a machine learn to read it, at least a first pass, the way modern translation models are built, and can it learn its dictionary without ever being given one?

The MTAAC / CDLI parallel corpus: 32,578 aligned Sumerian and English phrases from real cuneiform tablets, administrative and literary, in the transliteration Assyriologists use rather than raw cuneiform. The corpus authors are candid that it is noisy, so the project is scoped and framed the same honest way.

A hybrid that extends the standard NLP pipeline (text to sequences, padding, embeddings, seq2seq). Layer one, live now: a dictionary learned by IBM Model 1 word alignment, which discovers that udu means "sheep" and ki means "from" purely from which words keep travelling together. Layer two, next: an LSTM encoder-decoder that reads whole lines in context.

With no dictionary supplied, alignment alone recovers correct readings for hundreds of Sumerian words and glosses formulaic tablet lines sensibly. It also surfaces the hard part honestly: Sumerian homophones like gu (ox, talent, thread) trip a context-free gloss, which is exactly what the neural layer is built to fix. Type a line and watch it read, word by word, with confidence and alternate readings.