Tag: llm

  • Divergent thoughts on AI

    GlitchTip, the open source error monitoring tool I started, features no capital-fueled AI language on it’s home page. Our LLM wrapper isn’t changing your world, it doesn’t exist. Compare that to a competitor like Sentry’s seer, where the marketing features some fleshy, pained “Allied Mastercomputer” or AM watching your errors.

    Was that intentional to look so pained that it might get attention? I suppose it worked.

    Use the chat widget on the site, and I’ll respond slowly when I get around it it. You might think I’m a Luddite. I do have a Minimal Phone now, that I like. It’s less distracting. I can’t watch video on my phone. Reading slow is fine, but scrolling media is pained. It’s much less enticing for my kids to want to grab. It turns internet ads with motion into eink flashing nausea. Which I perversely enjoy hating. It’s ugly, but I’m glad the ad is rendered incomprehensible.

    In my day job (Not GlitchTip) I actually do a good bit of ai work. I mean both using ai and implementing it (Building wrappers). Not the fancy building of GPT systems, I understand only at a elementary level, maybe as much as I understand quantum physics.

    In spite of my technology reservations, I actually do like the idea of more tooling to automate coding. I’ve always felt drawn more to programming as a way to build something, while not really enjoying writing algorithms very much. I find some coding frustrating. Stop upgrading your packages, I hate your breaking changes.

    Last summer I built this unfinished media visualizer https://gitlab.com/bufke/disk-player using only ai tools. I just wanted to see if I could. I lack the attention to finish it, but it is kind of cool. I found the experiment successful, I could use my existing knowledge to let a LLM build something for me, only reviewing and guiding it.

    In the fall, I built django-vcache and django-vtasks to replace Celery for me with more asyncio friendly and lighter code. I used a combination of Gemini CLI and Claude+Zed. Working on open source, I can’t say I built some amazing $80k project with $200 of credits. All I could say is that I built a $0 money pit with $200 of credits and now I cannot afford coffee. Gemini pro $20/month pricing is reasonable and I paid for Zed pro mainly to support an open source project that I like. You should do this too, maybe for Zed or GlitchTip. My total budget is then capped $50/month, that’s less coffee but not no coffee for me.

    I use a Gemini Pro web conversation as a architect/planner and CLI/Zed as the doer. The web conversation is a playground for ideas and longer historical context. LLM context is kept lower as it’s not writing much code. I might have a good idea while on the subway and quickly update the conversation. The Minimal phone does has a nice physical keyboard for quick writing. To execute, I can have the architect LLM write a prompt for CLI/Zed. I can let CLI run while cooking. Then give a deep dive when merging, when I’m not distracted. I ended up writing many unit tests myself to ensure correctness, the models tend to write tests for the sake of testing and not proving anything. They are too full of mocks and misunderstanding. The better tests ground changes, avoiding one feature breaking another. Or the totally bonkers things LLM’s do sometimes. Early results for these projects are very promising. They perform well and are good for my half-distracted lifestyle. I don’t suspect I would consider a “make my own Celery” type project without them.

    I could imagine using voice controls for these tools. Maybe I talk to gemini while holding my baby. I get work done at all hours of the day, supervising it. Let it work alone at night. During dedicated work time, I review deeply and commit. Sounds like a hell AM might dream up. Total distraction. Total focus on a feeling of work. Unsure if it’s useful, effective, dangerous.

    I think I’ll avoid voice controls for now.

    Here’s a better use case. I maintain all of these test projects. https://gitlab.com/glitchtip/error-factories. They ensure the sentry-sdk works with GlitchTip. They auto update themselves with Renovate. They send events to a staging GlitchTip server every day. A LLM manages their maintenance when tests fail. This is a highly effective use of ai in my opinion. As a large language model, I—just kidding. Nothing in error factories is production code. And it’s boring code. In this one case, I think the ai usage is firmly more time with family and coffee.