https://pierce.dev Software engineer and writer en-us Tue, 14 Apr 2026 03:43:02 GMT https://pierce.dev/notes/the-grey-market-of-podcast-appearances/ https://pierce.dev/notes/the-grey-market-of-podcast-appearances/ Thu, 09 Apr 2026 00:00:00 GMT People are trading horses to get on podcasts. Notes https://pierce.dev/notes/the-way-i-travel/ https://pierce.dev/notes/the-way-i-travel/ Sun, 01 Mar 2026 00:00:00 GMT Strong opinions on travel, pretty strongly held. Travel https://pierce.dev/notes/fixing-slow-aws-uploads/ https://pierce.dev/notes/fixing-slow-aws-uploads/ Wed, 18 Feb 2026 00:00:00 GMT Double check your NVMe destinations Notes https://pierce.dev/notes/local-tools-should-still-use-vaults/ https://pierce.dev/notes/local-tools-should-still-use-vaults/ Tue, 10 Feb 2026 00:00:00 GMT Use 1Password with your Pydantic configurations. Notes https://pierce.dev/notes/we-solved-scratch-content-first/ https://pierce.dev/notes/we-solved-scratch-content-first/ Mon, 02 Feb 2026 00:00:00 GMT AI image generation solved from-scratch creation before editing, leaving us with infinite creative power but zero creative control. Notes https://pierce.dev/notes/starting-a-podcast-in-2025/ https://pierce.dev/notes/starting-a-podcast-in-2025/ Wed, 26 Nov 2025 00:00:00 GMT Starting a new podcast can feel late, but market timing matters less than making something meaningfully different. Notes https://pierce.dev/notes/being-late-but-still-being-early/ https://pierce.dev/notes/being-late-but-still-being-early/ Tue, 25 Nov 2025 00:00:00 GMT Living in San Francisco makes you feel late to every trend because your reference group is distorted. Notes https://pierce.dev/notes/automating-our-home-video-imports/ https://pierce.dev/notes/automating-our-home-video-imports/ Wed, 05 Nov 2025 00:00:00 GMT Digitizing decades of family videos from MiniDV and Hi8 tapes at lossless quality using dvrescue and FireWire for ~$300 instead of $4k. Notes https://pierce.dev/notes/adding-my-parents-to-tailscale/ https://pierce.dev/notes/adding-my-parents-to-tailscale/ Mon, 03 Nov 2025 00:00:00 GMT Setting up secure remote access to a home media server for family members using Tailscale's VPN and ACL permissions. Notes https://pierce.dev/notes/a-deep-dive-on-agent-sandboxes/ https://pierce.dev/notes/a-deep-dive-on-agent-sandboxes/ Fri, 26 Sep 2025 00:00:00 GMT How modern coding agents use OS-level sandboxing with macOS Seatbelt and Linux Landlock to balance powerful tool access with security. Notes https://pierce.dev/notes/language-servers-for-ai/ https://pierce.dev/notes/language-servers-for-ai/ Thu, 25 Sep 2025 00:00:00 GMT Why language servers are becoming more important for AI coding agents than for human developers as deterministic validators in correction loops. Notes https://pierce.dev/notes/my-simple-home-podcast-studio/ https://pierce.dev/notes/my-simple-home-podcast-studio/ Mon, 22 Sep 2025 00:00:00 GMT Building a home podcast studio from scratch with professional audio, 4K video recording, and remote guest capabilities for under $1k. Notes https://pierce.dev/notes/we-need-centralized-infrastructure/ https://pierce.dev/notes/we-need-centralized-infrastructure/ Tue, 12 Aug 2025 00:00:00 GMT Despite the distributed computing hype, centralized infrastructure often delivers better efficiency and economics when you run the actual numbers. Notes https://pierce.dev/notes/coercing-agents-to-follow-conventions-using-ast-validation/ https://pierce.dev/notes/coercing-agents-to-follow-conventions-using-ast-validation/ Mon, 11 Aug 2025 00:00:00 GMT Using LLMs to generate AST validators that enforce coding conventions and prevent agents from taking shortcuts in testing. Notes https://pierce.dev/notes/my-unified-theory-of-social-selling/ https://pierce.dev/notes/my-unified-theory-of-social-selling/ Mon, 11 Aug 2025 00:00:00 GMT How enterprise sales changes when you can't rely on email communication anymore. You turn to social media as a distribution channel. Notes https://pierce.dev/notes/my-personal-backup-strategy/ https://pierce.dev/notes/my-personal-backup-strategy/ Sat, 02 Aug 2025 00:00:00 GMT A dual-redundant backup system using rclone, Backblaze B2, and Docker to encrypt and sync data across multiple geographic regions. Notes https://pierce.dev/notes/july-updates-to-the-homelab/ https://pierce.dev/notes/july-updates-to-the-homelab/ Tue, 29 Jul 2025 00:00:00 GMT Upgrading from QNAP to Unifi UNAS for faster network storage and cleaner rack organization with patch panels and proper mounting hardware. Notes https://pierce.dev/notes/how-the-kv-cache-works/ https://pierce.dev/notes/how-the-kv-cache-works/ Sat, 26 Jul 2025 00:00:00 GMT An interactive explanation of how transformer models cache key-value pairs to avoid redundant calculations and speed up token generation. Notes https://pierce.dev/notes/httpx-is-the-right-way-to-do-web-requests-in-python/ https://pierce.dev/notes/httpx-is-the-right-way-to-do-web-requests-in-python/ Wed, 23 Jul 2025 00:00:00 GMT Why httpx is becoming the standard for Python HTTP requests with async support, reliable testing, and OpenAPI client generation. Notes https://pierce.dev/notes/reputation-is-becoming-everything/ https://pierce.dev/notes/reputation-is-becoming-everything/ Sun, 20 Jul 2025 00:00:00 GMT How AI-assisted interview tools are making remote screening unreliable and shifting hiring toward reputation-based systems. Notes https://pierce.dev/notes/building-a-kind-of-invisible-mac-app/ https://pierce.dev/notes/building-a-kind-of-invisible-mac-app/ Sun, 20 Jul 2025 00:00:00 GMT A technical exploration of how window-level capture protection works in macOS. Notes https://pierce.dev/notes/updated-knowledge-in-language-models/ https://pierce.dev/notes/updated-knowledge-in-language-models/ Sat, 19 Jul 2025 00:00:00 GMT When to bake knowledge into model weights versus using retrieval tools, and why experiential knowledge differs from factual knowledge. Notes https://pierce.dev/notes/making-an-ascii-animation/ https://pierce.dev/notes/making-an-ascii-animation/ Tue, 15 Jul 2025 00:00:00 GMT Breaking down the technical process behind Ghostty's terminal animation using color filtering, luminance mapping, and ASCII character density. Notes https://pierce.dev/notes/how-speculative-decoding-works/ https://pierce.dev/notes/how-speculative-decoding-works/ Sun, 13 Jul 2025 00:00:00 GMT Using a small draft model to generate speculative tokens that a large model verifies in parallel, achieving 2-4x inference speedup without quality loss. Notes https://pierce.dev/notes/under-the-hood-of-claude-code/ https://pierce.dev/notes/under-the-hood-of-claude-code/ Sat, 05 Jul 2025 00:00:00 GMT Reverse-engineering Claude Code's architecture by analyzing its minimized JavaScript to understand tool prompts, task decomposition, and error handling. Notes https://pierce.dev/notes/doing-things-because-theyre-easy-not-hard/ https://pierce.dev/notes/doing-things-because-theyre-easy-not-hard/ Thu, 03 Jul 2025 00:00:00 GMT How AI agents are changing the economics of side projects by making automation cheap enough to build tools you'd never justify manually. Notes https://pierce.dev/notes/go-ahead-self-host-postgres/ https://pierce.dev/notes/go-ahead-self-host-postgres/ Wed, 02 Jul 2025 00:00:00 GMT Self-hosting Postgres is simpler and cheaper than managed services suggest, with comparable reliability and better performance tunability. Notes https://pierce.dev/notes/speeding-up-sideeffects-with-jit-in-mountaineer/ https://pierce.dev/notes/speeding-up-sideeffects-with-jit-in-mountaineer/ Wed, 25 Jun 2025 00:00:00 GMT Using AST analysis to generate specialized render functions at runtime that only compute the state fields actually modified by a sideeffect. Notes https://pierce.dev/notes/firehot-for-hot-reloading-in-python/ https://pierce.dev/notes/firehot-for-hot-reloading-in-python/ Tue, 24 Jun 2025 00:00:00 GMT A hybrid hot reloading approach using Unix fork() to keep third-party imports cached while reloading project code for sub-second restart times. Notes https://pierce.dev/notes/misadventures-in-python-hot-reloading/ https://pierce.dev/notes/misadventures-in-python-hot-reloading/ Tue, 24 Jun 2025 00:00:00 GMT Why Python hot reloading is harder than JavaScript's and the tradeoffs between process restarts, selective invalidation, and hybrid approaches. Notes https://pierce.dev/notes/how-text-diffusion-works/ https://pierce.dev/notes/how-text-diffusion-works/ Mon, 23 Jun 2025 00:00:00 GMT How Google's Gemini Diffusion generates text by progressively unmasking tokens in parallel instead of autoregressive left-to-right generation. Notes https://pierce.dev/notes/the-tenacity-of-modern-llms/ https://pierce.dev/notes/the-tenacity-of-modern-llms/ Sat, 21 Jun 2025 00:00:00 GMT How modern LLMs iterate relentlessly through feedback loops with unit tests, trying different approaches until they find working solutions. Notes https://pierce.dev/notes/the-ergonomics-of-rails/ https://pierce.dev/notes/the-ergonomics-of-rails/ Mon, 16 Jun 2025 00:00:00 GMT Why Rails succeeded over PHP by eliminating repetitive work through conventions, integrated ORMs, and developer-first design philosophy. Notes https://pierce.dev/notes/how-language-servers-work/ https://pierce.dev/notes/how-language-servers-work/ Sat, 14 Jun 2025 00:00:00 GMT A technical deep dive into language server internals, from AST parsing to semantic analysis and rule engines that power IDE linting. Notes https://pierce.dev/notes/just-add-eggs/ https://pierce.dev/notes/just-add-eggs/ Sun, 08 Jun 2025 00:00:00 GMT Cracking the egg is as true in software as it is in cake mix. Notes https://pierce.dev/notes/unfortunately-seo-still-matters/ https://pierce.dev/notes/unfortunately-seo-still-matters/ Sun, 01 Jun 2025 00:00:00 GMT Everyone's saying SEO is done and dusted. It's over. GEO (generative engine optimization) is the name of the game now. Or maybe companies should stop writing bad quality stuff. Notes https://pierce.dev/notes/the-futility-of-human-only-web-requirements/ https://pierce.dev/notes/the-futility-of-human-only-web-requirements/ Sun, 01 Jun 2025 00:00:00 GMT AI agents may well become the main entities that touch the web. Too bad websites try to block them at every turn. Fundamentally the web's anti-bot defenses are solving the wrong problem. Who cares whether that's human or AI-mediated? Notes https://pierce.dev/notes/setting-up-input-leap/ https://pierce.dev/notes/setting-up-input-leap/ Thu, 29 May 2025 00:00:00 GMT InputLeap is an open source KVM that lets you share your keyboard and mouse with multiple machines over the network. Perfect for controlling rack-mounted homelab servers without a dedicated keyboard. Notes https://pierce.dev/notes/checking-in-on-waymo/ https://pierce.dev/notes/checking-in-on-waymo/ Wed, 28 May 2025 00:00:00 GMT In under two years cars went from what felt like some CV models and a detailed control system to driving better than most people. I don't know a single person that prefers to take Ubers anymore. If Waymos are any indication of how far we can push transformers, it's pretty damn far. Notes https://pierce.dev/notes/the-react-revolution/ https://pierce.dev/notes/the-react-revolution/ Sat, 03 May 2025 00:00:00 GMT React changed the engineering world when it walked on stage in May 2013. React didn’t just win the JavaScript wars; it changed how every major platform thinks about UI. Notes https://pierce.dev/notes/speeding-up-many-small-transfers-to-a-unifi-nas/ https://pierce.dev/notes/speeding-up-many-small-transfers-to-a-unifi-nas/ Wed, 23 Apr 2025 00:00:00 GMT Moving some of my old local ML projects to a NAS was taking an _unbearably_ long time. A folder with 25k small files was estimating a whole day to transfer. When bundled into a single tarfile it finished in 2 minutes flat. Notes https://pierce.dev/notes/quick-notes-on-swift-libraries/ https://pierce.dev/notes/quick-notes-on-swift-libraries/ Thu, 20 Mar 2025 00:00:00 GMT I've been dipping my toes back into the macOS development scene after a long hiatus of focusing only on the web. After being spoiled by the package ecosystem of Python+JS, I was wondering how different the Swift scene looks from the Objective-C scene of yore. Notes https://pierce.dev/notes/ai-engineering-is-a-different-animal/ https://pierce.dev/notes/ai-engineering-is-a-different-animal/ Sat, 15 Mar 2025 00:00:00 GMT Great engineers can see most of the major roadblocks in the way because they can visualize how to get from Point A to Point B. You make progress day by day. You pull some all-nighters. And before you know it you have enough features to make a product. AI is not like that. Notes https://pierce.dev/notes/san-francisco/ https://pierce.dev/notes/san-francisco/ Thu, 06 Mar 2025 00:00:00 GMT Skip the Golden Gate photos and hit the coastal trails, farmers markets, and foggy beaches where locals actually spend their weekends. Travel https://pierce.dev/notes/debugging-a-mountaineer-rendering-segfault/ https://pierce.dev/notes/debugging-a-mountaineer-rendering-segfault/ Mon, 03 Mar 2025 00:00:00 GMT Debugging a V8 memory allocation segfault caused by CPU PKU flags when initializing the platform on secondary threads instead of the main thread. Notes https://pierce.dev/notes/local-network-config-on-macos/ https://pierce.dev/notes/local-network-config-on-macos/ Fri, 28 Feb 2025 00:00:00 GMT The "Local Network" Privacy setting on macos needs a redesign. Notes https://pierce.dev/notes/building-our-home-network/ https://pierce.dev/notes/building-our-home-network/ Sun, 09 Feb 2025 00:00:00 GMT Notes on building our 10gbps home network, from choosing Sonic as our ISP to a blinking server rack in the corner. Notes https://pierce.dev/notes/introducing-envelopedev/ https://pierce.dev/notes/introducing-envelopedev/ Tue, 12 Nov 2024 00:00:00 GMT Envelope is a new package registry for private python packages. It started as a side project just to host my own packages. After using it for the last year, I realized that it could be useful for others as well. Notes https://pierce.dev/notes/legacy-code-and-ai-copilots/ https://pierce.dev/notes/legacy-code-and-ai-copilots/ Thu, 11 Jul 2024 00:00:00 GMT In addition to the core language, code LLMs also have to interplay with an ecosystem of constantly changing dependencies. These package versions constantly change in features, with different functions and syntax. What are some long-term approaches to making coding assistants more aware of the package ecosystem? Notes https://pierce.dev/notes/typehinting-from-day-zero/ https://pierce.dev/notes/typehinting-from-day-zero/ Thu, 11 Jul 2024 00:00:00 GMT Static typehinted languages can make us lazy about adding types at the right time. We have all the context when we start a new project, but as we increase complexity and focus on other things that context wains. Rewards compound from typing on day zero. Notes https://pierce.dev/notes/generating-database-migrations-with-acyclic-graphs/ https://pierce.dev/notes/generating-database-migrations-with-acyclic-graphs/ Wed, 03 Jul 2024 00:00:00 GMT Mountaineer 0.5.0 introduced database migration support, so you can now upgrade production databases directly from the CLI. It generates SQL for you automatically instead of writing manual table migrations, and removes the need for third party packages to support the same functionality. Let's dive into the details of how we implemented the engine. Notes https://pierce.dev/notes/lofoten/ https://pierce.dev/notes/lofoten/ Sat, 29 Jun 2024 00:00:00 GMT Midnight sun, camper van living, and fishing for cod in the Arctic islands where summer never ends. Travel https://pierce.dev/notes/mountaineer-v01-webapps-in-python-and-react/ https://pierce.dev/notes/mountaineer-v01-webapps-in-python-and-react/ Tue, 27 Feb 2024 00:00:00 GMT Today I'm really excited to open source a beta of Mountaineer, an integrated framework to quickly build webapps in Python and React. It's initial goals are quite humble: make it really fun to design systems with these two languages. Notes https://pierce.dev/notes/constraining-llm-outputs/ https://pierce.dev/notes/constraining-llm-outputs/ Tue, 20 Feb 2024 00:00:00 GMT LLMs are by definition probabilistic; for each new input, they sample from a new distribution. Even the best prompt or finetuning will minimize (but not fully resolve) the chance that they give you output you don't expect. This is unlike a traditional application API, where the surface area is known and the fields have a guaranteed structure. Notes https://pierce.dev/notes/passthrough-above-all/ https://pierce.dev/notes/passthrough-above-all/ Thu, 15 Feb 2024 00:00:00 GMT In the Vision Pro, there's sometimes a conflict between the window's existence and your own passthrough reality. Try to place one in a room and then walk through a doorway, peeking back at the room from within the door frame. Practically speaking, it's better to keep the reality of what people are actually seeing than to keep the reality of the augmented reality. Notes https://pierce.dev/notes/accuracy-in-kudos/ https://pierce.dev/notes/accuracy-in-kudos/ Wed, 14 Feb 2024 00:00:00 GMT Why celebrating every small win dilutes the impact of truly meaningful accomplishments in workplace culture. Notes https://pierce.dev/notes/how-quick-we-are-to-adapt/ https://pierce.dev/notes/how-quick-we-are-to-adapt/ Tue, 06 Feb 2024 00:00:00 GMT Most of the people I know in San Francisco have used a Waymo at least once. Many friends of mine swear by them. The fact they're self driving doesn't really enter into the equation: they just prefer the product they're being offered when they're picked up. Notes https://pierce.dev/notes/the-curious-case-of-lm-repetition/ https://pierce.dev/notes/the-curious-case-of-lm-repetition/ Mon, 22 Jan 2024 00:00:00 GMT I was doing some OSS benchmarking over the weekend and was running into an odd issue. Some families of models would respond with near-gibberish, even with straightforward prompt inputs. This is a debugging session for LLM repetition. Notes https://pierce.dev/notes/costa-rica/ https://pierce.dev/notes/costa-rica/ Sun, 14 Jan 2024 00:00:00 GMT Cloud forests, surf towns, and learning that birding tours start at 5:30am for good reason. Travel https://pierce.dev/notes/debugging-chrome-extensions-with-system-level-logging/ https://pierce.dev/notes/debugging-chrome-extensions-with-system-level-logging/ Tue, 19 Dec 2023 00:00:00 GMT Extensions are basically mini web applications these days, just with access to a `chrome` global variable that can interact with some browser-level functionality. Aside from that - it's all familiar. That extends to the debugging experience. Since extensions run in the regular V8 Chrome runtime, Chrome exposes the same debugging tools that you're used to on the web. Notes https://pierce.dev/notes/speeding-up-runpod/ https://pierce.dev/notes/speeding-up-runpod/ Mon, 18 Dec 2023 00:00:00 GMT One issue I've occasionally observed on Runpod is varying runtime performance box-to-box. My working mental model of VMs is that you have full control of your allocation; if you've been granted 4 CPUs you get the ability to push 4 CPUs to the brink of capacity. Of course, the reality is a bit more murky depending on your underlying kernel and virtual machine manager, but usually this simple model works out fine. Notes https://pierce.dev/notes/inline-footnotes-with-html-templates/ https://pierce.dev/notes/inline-footnotes-with-html-templates/ Sun, 17 Dec 2023 00:00:00 GMT I couldn’t write without footnotes. Or at least - I couldn't write enjoyably without them. They let you sneak in anecdotes, additional context, and maybe even a joke or two. They're the love of my writing life. For that reason, I wanted to get them closer to the content itself through inline footnotes. Notes https://pierce.dev/notes/parsing-common-crawl-in-a-day-for-60/ https://pierce.dev/notes/parsing-common-crawl-in-a-day-for-60/ Thu, 14 Dec 2023 00:00:00 GMT In addition to forming a bulk of the foundation of modern language models, there's a ton of other data buried within Common Crawl. Incoming and external links to websites, referral codes, leaked data. If it's public on the Internet, there's a good chance CC has it somewhere within its index. Here we parse all of common crawl in a day, on the cheap. Notes https://pierce.dev/notes/an-era-of-rich-cli/ https://pierce.dev/notes/an-era-of-rich-cli/ Fri, 03 Nov 2023 00:00:00 GMT How modern frameworks like Textual and Rich are bringing interactive UI experiences to the terminal with CSS-inspired layouts and component models. Notes https://pierce.dev/notes/all-or-nothing-with-remote-work/ https://pierce.dev/notes/all-or-nothing-with-remote-work/ Mon, 04 Sep 2023 00:00:00 GMT Companies can add tech and policies to faciliate remote work. Even notoriously secretive companies (consulting groups, lawfirms, etc) were able to configure remote working tools to their infosec policies. And they adapted very quickly when productivity was on the line. Notes https://pierce.dev/notes/the-next-10-years/ https://pierce.dev/notes/the-next-10-years/ Thu, 24 Aug 2023 00:00:00 GMT Personal notes for where we're headed over the next 10 years. While the future is never written in stone, I'm 90% sure of these outcomes. Past a decade, my confidence diminishes significantly. Notes https://pierce.dev/notes/adding-wheels-to-flash-attention/ https://pierce.dev/notes/adding-wheels-to-flash-attention/ Sun, 20 Aug 2023 00:00:00 GMT flash-attention is a low level implementation of exact attention. Unlike torch, which processes attention multiplications in sequence, `flash-attention` combines the operations into a fused kernel, which can speed up execution by 85%. And since attention is such a core primitive of most modern language models, it makes for much faster training and inference across the board. It now has an install time that's just as fast. Notes https://pierce.dev/notes/llms-as-interdisciplinary-agents/ https://pierce.dev/notes/llms-as-interdisciplinary-agents/ Fri, 26 May 2023 00:00:00 GMT The real breakthrough with large language models might not be exceeding human levels of performance in a discrete task. Perhaps it's enough that they can approach human level performance in a variety of tasks. There might be more whitespace in intersectional disciplines than aiming for true expert status in any one. Notes https://pierce.dev/notes/new-zealand/ https://pierce.dev/notes/new-zealand/ Tue, 23 May 2023 00:00:00 GMT Working Pacific hours across both islands while chasing vineyards, empty beaches, and dolphins in the sounds. Travel https://pierce.dev/notes/representations-in-autoregressive-models/ https://pierce.dev/notes/representations-in-autoregressive-models/ Thu, 11 May 2023 00:00:00 GMT One of my more memorable CV lectures in college opened with a declaration: representations in machine learning are everything. With a good enough representation, everything is basically a linear classifier. Focus on the representation, not on the network. Notes https://pierce.dev/notes/lets-talk-about-siri/ https://pierce.dev/notes/lets-talk-about-siri/ Fri, 28 Apr 2023 00:00:00 GMT Last weekend I spent some serious time with Siri for the first time in a couple years. A lot has changed since I last took a look. Since iOS 15, all NLU processing is done locally on device. There's a local speech-to-text model, a local natural-language-understanding module, and a local text-to-speech model. All logic appears hard-coded and baked into the current iOS version. Notes https://pierce.dev/notes/minimum-viable-public-infrastructure/ https://pierce.dev/notes/minimum-viable-public-infrastructure/ Thu, 27 Apr 2023 00:00:00 GMT You can't iterate when you're building huge things. You also can't tolerate failure in the same way. You don't want a bridge constructed in a month only to fall down the year after. The bulk of the bureaucracy for infrastructure is making sure projects meet this bar of safety; safe to use, safe to be around, and safe for the environment. There's no such thing as MVP Public Infrastructure. Notes https://pierce.dev/notes/reasoning-vs-memorization-in-llms/ https://pierce.dev/notes/reasoning-vs-memorization-in-llms/ Thu, 13 Apr 2023 00:00:00 GMT By virtue of their training objective, LLMs are optimized to model language and minimize the perplexity of examples. Memorization of input facts is an expected biproduct of this pipeline. General reasoning skills are the more unexpected emergent property. Notes https://pierce.dev/notes/automatically-migrate-enums-in-alembic/ https://pierce.dev/notes/automatically-migrate-enums-in-alembic/ Thu, 30 Mar 2023 00:00:00 GMT If you're using SQLAlchemy as your database ORM, there's a good chance you're using Alembic to migrate across revisions. Alembic doesn't support enums out of the box. Keep enum values in code synced up with database values. Notes https://pierce.dev/notes/greater-sequence-lengths-will-set-us-free/ https://pierce.dev/notes/greater-sequence-lengths-will-set-us-free/ Mon, 20 Mar 2023 00:00:00 GMT GPT-4 represents the latest leap in LLM sequence length. Doubling down on longterm dependencies might be the advance we need for real business value and machines that operate closer to humans. Notes https://pierce.dev/notes/on-learning-to-ski/ https://pierce.dev/notes/on-learning-to-ski/ Wed, 08 Mar 2023 00:00:00 GMT Common wisdom says children explore while adults exploit. At some point, we tend to transition from one to the other - perhaps because of risk intolerance, time limitations, or sheer laziness. I learned how to ski this year, which was the first new sport I've picked up in at least a decade. Some thoughts on learning new things and throwing yourself down mountains in the process. Notes https://pierce.dev/notes/dolomites/ https://pierce.dev/notes/dolomites/ Tue, 07 Mar 2023 00:00:00 GMT Skiing the Sellaronda loop and eating pasta on mountain refugios during a low-snow year that still delivered. Travel https://pierce.dev/notes/using-grpc-with-node-and-typescript/ https://pierce.dev/notes/using-grpc-with-node-and-typescript/ Thu, 16 Feb 2023 00:00:00 GMT Most of the grpc docs use the dynamic approach - I assume for ease of getting started. The main pro to dynamic generation is faster prototyping if the underlying schema changes, since you can hot reload the server/client. But one key downside includes not being able to typehint anything during development or compilation. For production use compiling it down to static code is a must. Notes https://pierce.dev/notes/opportunity-years/ https://pierce.dev/notes/opportunity-years/ Wed, 15 Feb 2023 00:00:00 GMT The last few months have been tough for a lot of people. Layoffs, down rounds, and bankruptcies jolt the expected progression of life. Decisions that were within grasp are now no longer. Despite the environment, people in tech are more optimistic than the media might lead you to believe. Notes https://pierce.dev/notes/buzzword-peaks-and-valleys/ https://pierce.dev/notes/buzzword-peaks-and-valleys/ Tue, 14 Feb 2023 00:00:00 GMT It's 2023 and once again, we are all in on AI. This is thanks in part to the cultural phenomena that is ChatGPT. Many companies are racing to deploy AI models (generative where possible) just to put it on their slide deck. Like clockwork, three years later, we've reverted back to AI. It sometimes feels like we're back in 2017. Notes https://pierce.dev/notes/buenos-aires/ https://pierce.dev/notes/buenos-aires/ Tue, 10 Jan 2023 00:00:00 GMT Celebrating Argentina's World Cup victory with 50,000 strangers while learning the blue market peso economy. Travel https://pierce.dev/notes/network-routing-interaction-on-macos/ https://pierce.dev/notes/network-routing-interaction-on-macos/ Mon, 02 Jan 2023 00:00:00 GMT There are a series of resolution layers governing DNS, IP, and port routing on OSX. Included are notes on the different routing utilities supported locally, specifically using /etc/hosts, ifconfig, pfctl, and /etc/resolver. Notes https://pierce.dev/notes/independent-work-november-recap/ https://pierce.dev/notes/independent-work-november-recap/ Thu, 22 Dec 2022 00:00:00 GMT A progress update on my second month of independent work. Notes https://pierce.dev/notes/debugging-slow-pytorch-training-performance/ https://pierce.dev/notes/debugging-slow-pytorch-training-performance/ Wed, 21 Dec 2022 00:00:00 GMT A deep dive into debugging slow GPU utilization in a pytorch lightning training pipeline. Some tricks with SimpleProfiler and DatasetWrapper to help you debug your dataloader woes. Notes https://pierce.dev/notes/the-provenance-of-copy-and-paste/ https://pierce.dev/notes/the-provenance-of-copy-and-paste/ Mon, 19 Dec 2022 00:00:00 GMT Copy and paste is ubiquitous. A topic that receives less attention, however, is the provenance of data that flows into and out of your clipboard. I often find myself going through documents that I've written or were written by colleagues. I almost inevitably have to wonder where in the world some of the data came from. A thought experiment for a copy and paste implementation that retains a history chain going back to the original source. Notes https://pierce.dev/notes/debugging-tips-for-neural-network-training/ https://pierce.dev/notes/debugging-tips-for-neural-network-training/ Fri, 16 Dec 2022 00:00:00 GMT Practical notes for debugging more complicated training pipelines and architectures, informed by pure research and productionalizing models in industry. This guide has a bias towards debugging large language models. Notes https://pierce.dev/notes/patagonia/ https://pierce.dev/notes/patagonia/ Fri, 09 Dec 2022 00:00:00 GMT Ten days on the W Trek where your feet hurt for weeks but the glaciers make it worth every step. Travel https://pierce.dev/notes/santiago/ https://pierce.dev/notes/santiago/ Thu, 08 Dec 2022 00:00:00 GMT Wine bars and empanadas in a city that deserves more than being treated as a Patagonia layover. Travel https://pierce.dev/notes/my-2022-digital-travel-kit/ https://pierce.dev/notes/my-2022-digital-travel-kit/ Mon, 21 Nov 2022 00:00:00 GMT Five months on the road with an M1 MacBook, Canon R5, and the realization that laptop stands aren't worth the weight. Travel https://pierce.dev/notes/aws-vs-gcp-gpu-availability-v2/ https://pierce.dev/notes/aws-vs-gcp-gpu-availability-v2/ Mon, 14 Nov 2022 00:00:00 GMT A revised comparison between GPU availability for AWS and GCP. Includes some internal strategies for GCP request allocation. Updated benchmarking numbers. Notes https://pierce.dev/notes/independent-work-october-recap/ https://pierce.dev/notes/independent-work-october-recap/ Sat, 05 Nov 2022 00:00:00 GMT It's been a month since going full time on my own thing. In some ways I'm surprised by how natural the transition has been. This is a short progress update on the first month of going independent. Finished a first launch of GrooveProxy with some progress on Popdown. Notes https://pierce.dev/notes/planning-patagonia/ https://pierce.dev/notes/planning-patagonia/ Wed, 02 Nov 2022 00:00:00 GMT The logistics of booking Torres del Paine campsites before they sell out and choosing between Chilean and Argentine routes. Travel https://pierce.dev/notes/relationship-modeling/ https://pierce.dev/notes/relationship-modeling/ Tue, 25 Oct 2022 00:00:00 GMT Given the pandemic's isolation of friends and friend groups, I've been thinking a lot about relationships. Which ones fulfill, which ones entertain, and which ones are resilient to strain. Why do we spend so much time talking about the past or trying to predict the future? Notes https://pierce.dev/notes/the-power-of-status-updates/ https://pierce.dev/notes/the-power-of-status-updates/ Wed, 19 Oct 2022 00:00:00 GMT There's a reason why dashboards have become increasingly common over the last decade. Hearing from people with more context can immediately dissolve fears. In that way trains have a lot to do with status pages. Notes https://pierce.dev/notes/a-new-chapter/ https://pierce.dev/notes/a-new-chapter/ Thu, 13 Oct 2022 00:00:00 GMT Last week I said goodbye to my colleagues at Globality after five years on their engineering team. It's hard to believe it's been so long. I still remember my first day perfectly - no laptop, no desk, not even a manager to greet me. I ended up writing my first PR on a personal computer in the kitchenette. I've been putting a lot of thought into what I want to focus on next. Here's my current list. Notes https://pierce.dev/notes/give-my-library-a-coffee-shop/ https://pierce.dev/notes/give-my-library-a-coffee-shop/ Wed, 28 Sep 2022 00:00:00 GMT Libraries might be one of the greatest assets in modern America. They're free, have an extensive selection, provide technological support, dot cities and rural counties alike, and are often beautifully architected. Their physical spaces are also increasingly underutilized. Notes https://pierce.dev/notes/aws-vs-gcp-gpu-availability-v1/ https://pierce.dev/notes/aws-vs-gcp-gpu-availability-v1/ Wed, 21 Sep 2022 00:00:00 GMT Cloud compute is backed by physical servers. And with the chip shortage of CPUs and GPUs those resources are more limited than ever. After encountering some reliability issues with on-demand provisioning of GPU resources on Google Cloud, I put together a benchmarking harness to test AWS vs. GCP availability. Notes https://pierce.dev/notes/switzerland/ https://pierce.dev/notes/switzerland/ Sat, 10 Sep 2022 00:00:00 GMT Flawless trains, Alpine scrambles with rope assists, and cow bells echoing across valleys. Travel https://pierce.dev/notes/headfull-browsers-beat-headless/ https://pierce.dev/notes/headfull-browsers-beat-headless/ Wed, 07 Sep 2022 00:00:00 GMT Twenty years ago a simple curl would open up the world. HTML markup was largely hand designed so id and name attributes were easily interpretable and parsable. Now most sites render dynamic content or use template defined class tags to define the styling of the page. Building a headfull browser container to more easily deploy and debug Chromium in a remote cluster. Notes https://pierce.dev/notes/webcrawling-tradeoffs/ https://pierce.dev/notes/webcrawling-tradeoffs/ Tue, 06 Sep 2022 00:00:00 GMT A couple of years ago I built our internal crawling platform at Globality, which needed to be capable of scaling to billions of pages each crawl. The two main types of crawlers that are deployed in the wild are typically raw or headless. We ended up implementing a hybrid architecture. Hybrid crawling can make use of the strengths of both while trying to minimize their weaknesses. Notes https://pierce.dev/notes/copenhagen/ https://pierce.dev/notes/copenhagen/ Fri, 02 Sep 2022 00:00:00 GMT Self-driving metro trains, canal swimming, and cardamom buns that beat Paris pastries. Travel https://pierce.dev/notes/busses-can-fool-me-thrice/ https://pierce.dev/notes/busses-can-fool-me-thrice/ Tue, 30 Aug 2022 00:00:00 GMT Public transit is often framed as necessary philanthropy for cities. It cuts down on cars and pollution at the expense of convenience. If people can more efficiently get to their destination by other means, they will. This is the wrong way to look at things. For public transit to really work, it needs trust. The main KPI for a transit system has to be adherence to schedule. Notes https://pierce.dev/notes/the-transit-ratio/ https://pierce.dev/notes/the-transit-ratio/ Sun, 28 Aug 2022 00:00:00 GMT Decisions of how to get from one place to another aren't made in a vacuum. Notes https://pierce.dev/notes/programatic-money/ https://pierce.dev/notes/programatic-money/ Sun, 28 Aug 2022 00:00:00 GMT Cryptocurrency's real innovation isn't in replacing traditional currency, but in making money programmable. Institutional trust can still have an API. Notes https://pierce.dev/notes/amsterdam/ https://pierce.dev/notes/amsterdam/ Mon, 22 Aug 2022 00:00:00 GMT A weekend layover that packed in Michelin-star greenhouses, natural wine bars, and realizing bike infrastructure can transform cities. Travel https://pierce.dev/notes/paris/ https://pierce.dev/notes/paris/ Thu, 18 Aug 2022 00:00:00 GMT Working Pacific hours through Paris heat waves when Parisians flee the city and UPS loses your packages for sport. Travel https://pierce.dev/notes/falling-for-kubernetes/ https://pierce.dev/notes/falling-for-kubernetes/ Sun, 07 Aug 2022 00:00:00 GMT I default to bare metal where I can. But recently I had to adopt a more complicated server management solution. And after a couple months of building for kubernetes, I must admit I'm falling for it more every day. Notes https://pierce.dev/notes/cascais/ https://pierce.dev/notes/cascais/ Wed, 03 Aug 2022 00:00:00 GMT Mopeds to Sintra castles, coworking for $180/month, and discovering that kite surfing beaches exist outside Hawaii. Travel https://pierce.dev/notes/content-that-im-obsessed-with/ https://pierce.dev/notes/content-that-im-obsessed-with/ Tue, 02 Aug 2022 00:00:00 GMT A constantly updating collection of content that I highly recommend to others. Movies, TV, and Books. Updated occasionally if something has staying power of more than 3 months. Notes https://pierce.dev/notes/remote-work-is-a-better-tourism/ https://pierce.dev/notes/remote-work-is-a-better-tourism/ Thu, 09 Jun 2022 00:00:00 GMT Over the pandemic I've been able to work from a variety of places. I've vacationed to most before. In almost all cases, I've vastly preferred working there. It gives you the encouragement to do what locals do. You're way more likely to meet people who live there if you engage them where they're most likely to be doing work and living lives themselves. Notes https://pierce.dev/notes/the-new-opportunity-in-travel/ https://pierce.dev/notes/the-new-opportunity-in-travel/ Sun, 29 May 2022 00:00:00 GMT There is going to be a new class of travel option: working by day and socializing by night. This model upends traditional tourist activities since it encourages a participation in local cultural life, like the working professionals that live in that city full time. Notes https://pierce.dev/notes/labor-markets-calibrate-satisfaction/ https://pierce.dev/notes/labor-markets-calibrate-satisfaction/ Tue, 10 May 2022 00:00:00 GMT What explains the differing pay between talented people in different careers? Something is clearly lost in our typical conversation about what a salary includes. Notes https://pierce.dev/notes/installing-fasttext-on-an-m1-mac/ https://pierce.dev/notes/installing-fasttext-on-an-m1-mac/ Thu, 05 May 2022 00:00:00 GMT We rely on FastText in some of our NLP microservices. Since upgrading to an M1 Macbook, these dependencies have failed to build wheels. Notes https://pierce.dev/notes/architecting-a-blog/ https://pierce.dev/notes/architecting-a-blog/ Tue, 04 Jan 2022 00:00:00 GMT An obligatory post on blog architecture. I started focusing more on writing this year and wanted to rethink my workflow to make it a bit more frictionless. I started with the writing experience that I wanted in my IDE and moved on to the markdown compilation tooling. Notes https://pierce.dev/notes/write-where-you-are/ https://pierce.dev/notes/write-where-you-are/ Mon, 03 Jan 2022 00:00:00 GMT Publishing has always been my bottleneck. During stints on Wordpress or Medium, I was overly focused on how articles looked that it often got in the way of what they said. This year I want to change that trend. Notes https://pierce.dev/notes/treat-engineers-as-users/ https://pierce.dev/notes/treat-engineers-as-users/ Thu, 30 Dec 2021 00:00:00 GMT It's an underemphasized asset of successful engineering startups: they make developing enjoyable. More companies need to follow their lead and treat their internal teams like users. Give them a UX that they can enjoy. Notes https://pierce.dev/notes/scoping-an-ml-feature/ https://pierce.dev/notes/scoping-an-ml-feature/ Mon, 26 Apr 2021 00:00:00 GMT Most confusion when building ML features comes at the beginning of a project. The goals are vague, the data isn’t in the expected format, or the metrics are ill-defined. This is a key place for product managers to articulate user needs in a way that machine learning researchers can translate into a well-defined research problem. Notes https://pierce.dev/notes/ai-needs-a-better-definition/ https://pierce.dev/notes/ai-needs-a-better-definition/ Wed, 14 Apr 2021 00:00:00 GMT People label AI as anything and everything these days. You have search systems, you have process automation, you have spam filters. If motion activated supermarket doors were invented today, I guarantee they’d be branded AI too. Notes https://pierce.dev/notes/nfts-are-nothing-new/ https://pierce.dev/notes/nfts-are-nothing-new/ Mon, 05 Apr 2021 00:00:00 GMT NFT's have exploded into mainstream conversation over the last few weeks. Like with everything in crypto, you have strong bulls and equally strong bears on the investment thesis. Are the principles behind NFTs anything new? And what can collector culture tell us about the investment opportunities with these new tokens? Notes