I just returned from the AI Engineer SF World’s Fair 2025 and figured I’d recap my learnings with a focus on Software engineering with AI.

TL;DR

$ cat ideas | specs | ( parallel agents &) | review | profit

Learning #1 : AI Coding Agents are Truly everywhere

Being at its fourth edition, there is definitely a trend in AI coding becoming a topic on its own at the event. Obviously, the conference had the usual LLM models, RAG, graphRAG, and evals tracks, but this time it had a dedicated SWE-Agents track. It reflects the rising interest in coding with AI from the audience as presented by @swyx as well as the rising focus in the market as software engineering has shown itself to be a large success story for AI.

Every major AI coding tool was showcasing its new agent mode. They all bring their own narrative, but in essence, they all say autocompletion has been replaced with prompting, MCP tooling, and agents.

Learning #2 : Using AI Tools Like You Did 6 Months Ago Is a Mistake

Running agents also have an impact on coding interfaces. Some takeaways:

Learning #3 - Specs are the new code

Code editors now pretty much have their own flavor of rules.md and allow you to add markdown files into a prompt. This removes the tedious typing in the prompt and allows you to build up a consistent set of requirements.

It was great to see Sean Grove, member of technical staff at OpenAI not only talk about ModelSpecs but also going all in on Specification Driven Development. The talk is chuck full of insights, here’s a brief summary from his talk “The new code: specs write once, run everywhere” (it’s a must watch!):

This talk feels like a well articulated presentation on the 2nd AI Native Dev pattern : From implementation to intent.

Learning #4 - Agents … to the cloud!

With Agents moving to the cloud, there was also a rise in sandboxed execution vendors in the space. This is different from serverless execution systems as agents are typically longer running, can be suspended/resumed, and need streaming output.

Much attention was given to the security issues from executing unvetted code and content and requiring rigorous permissions settings: Can it go out on the internet? What internal sources can it access? OpenAI Codex by default disables that access and allows you to set the right access levels.

Learning #5: Parallel execution means Parallel exploration

Going to the cloud solves the need for extra desktop horsepower:

This matches the 3rd AI Native Dev pattern from Delivery to Discovery quite well: as execution and tokens get so cheap, we can explore different hypotheses.

Learning #6 - Is CI/CD shifting left?

With more and more code being produced, we’ll need better tooling for reviewing Pull Requests. Many AI Quality assurance tools were present as well:

Imblue’s CTO Josh Albrecht emphasized in his talk that the specs having strict guidelines also help in catching issues earlier in the value stream. Catching them at PR time is already more costly compared to catching them while coding. He referred to this as shifting left the testing of AI coding: If you think of the infinity DevOps loop, there is a closer agentic loop happening before the typical Test and QA running right in the developer’s IDE.

Learning #7 : How many X’s will AI deliver?

A common question around AI coding tools is how to measure their impact. While claims of 5x, 10x, or even 20x productivity gains are often seen on social media, many companies aren’t experiencing results anywhere near that. Why is that the case?

In his session, Yegor Denisov-Blanch (from Ghost Engineer fame) pointed out research that indicates the success with AI depends on:

Now these are technical factors, but don’t forget it also depends on your team and company technical maturity, as Fernando Cornago SVP at Adidas, points out in his ETLS talk.

Closing thoughts

I’ll certainly be back next time!