This issue talks about testing in production, how to make staging more prod-like, has a very healthy dose of AI stuff, and more!
Always With the Learning
Cindy Sridharan has a great series of posts about testing in production. Worth the long read, especially for those of us with distributed systems and microservices.
Shreya Ramesh writes about how Slack load tests using a custom-built tool.
This post about creating sufficient test data in staging really resonated with me, who normally uses the same test data conventions for creating orders and such. AI makes this so much easier now too.
Charity Majors’ keynote from 2019 about testing in production goes into observability as well as why it’s a good thing to test in production and not (just) staging.
Memesis
QEs Just Wanna Have Fun
Are you a fan of Dungeon Crawler Carl and wish you could have achievements with a bit of snark for things you do? Wait no longer! You can generate achievements for the mundane to the spectacular with this fun little webapp . As an example, I used “thrice” in a Slack message, and… it had thoughts.
Model Citizen
Jason Gorman gives a good mnemonic for great context engineering: C.R.E.S.S. Read about it here .
Simon Prior developed a manifesto for AI Quality Leadership. It’s pretty compelling stuff. Check out his entire series on Intelligent Quality Leadership.
Keeping up with testing AI-generated code is a problem the quality community has been talking about for a couple years now. Here’s a recent and thoughtful analysis of the problem and potential solutions by Lilia Abdulina and Vitaly Sharovatov. A very powerful line (in the research problem portion of it) for having comprehension of the code without understanding the intent: “The codebase becomes a museum of decisions whose context is gone.” This is similar to the principle of Chesterton’s fence , in my opinion, which I come back to whenever we talk about refactoring legacy code.
Maaret Pyhäjärvi has a really solid take on exploratory testing in the age of AI:
Angie Jones writes about how Oracle approaches AI memory (spoiler: AI database!). She addresses the many problems and ways of looking at memory too.
source: Simon Willson’s Weblog
This list of AI worries from a testing perspective is a little dated (written in October 2024, how is that dated??? just wild) but still quite relevant. Other concerns the community is talking about that aren’t on this list include but are not limited to:
- guardrails
- logic traps
- AI interacting with other AI
This is not to dissuade from using AI, merely to inform, educate, and hopefully create more thoughtful usage.
Odds and Ends in Engineering
As a follow-up to falsehoods about time from many issues ago, my brilliant husband pulled together all the falsehoods about… everything… in this falsehoods collection on Github . As we address things like names, numbers, timezones, everything we know about the world, this is a good resource for debunking what we know to be true.