Build to learn, build to earn — the loop AI hasn't closed [Joel Schwarzmann]

James Mulligan00:01

All right — Joel, introduce yourself. Who are you, and where do you work?

Joel Schwarzmann00:07

My name is Joel. I'm a product manager at a company called PhysicsX. We do all sorts of fun things with physics simulations, AI and ML, and try to answer some hard engineering questions with all of that.

James Mulligan00:20

Nice. And what's your role within PhysicsX — what are you focused on day to day?

Joel Schwarzmann00:26

I'm looking at a bunch of things. A lot of it is about building an enterprise-grade platform — all the stuff you need for that in terms of permissions and data, the bits that are fundamental to making the platform work. But that's also where agents fit in. Agents also need permissions and data, and that's a space that hasn't been thought through properly yet.

James Mulligan00:49

That's fascinating — and something I suspect a lot of software builders are thinking about right now. There's that decision: do you make your technology agentic and let agents control it, or not? And if you do decide to, what questions do you need to ask? Because it's not as simple as asking Claude to look at your product and use it for you, is it?

Joel Schwarzmann01:11

Some of these things are being invented as we speak; some are just standard software-engineering best practice. There was a joke when MCP came out that the "S" in MCP stood for security. That was true for a time — the authentication spec came together mid last year and is now well adopted and understood. That stuff is moving. But there are open questions about how you do permissions for an agent that needs to look at sensitive data. How do you build guardrails around that? How long does it have access? Who does it pretend to be — or is it a system role? There are lots of open, unanswered questions there. And if I were in a security role: how do I balance all the doors we're opening to experiment with the actual security risks I'm supposed to be managing? It's an interesting time.

James Mulligan02:10

No doubt. And as a product manager — your background isn't just product, is it? You're also an engineer by trade.

Joel Schwarzmann02:17

Yeah, I spent time doing data engineering and machine-learning engineering.

James Mulligan02:21

So from your perspective: if you were a PM today thinking "I have a piece of technology and I want an agent to use it," what are the big questions someone should be answering first?

Joel Schwarzmann02:35

What I've found recently is that the core product toolkit is really well suited to building products — and the fact that we can now build things quicker is interesting. A product requirements document is still really useful. Double-diamond design research is still really useful. A lot of good thinking has gone into all of that. The fact that you can now generate the content very quickly is both good and bad — everyone knows what slop looks like, you can see it a mile off these days. And there's the question of what an MVP even means now. You still want to build a skateboard, then a bike, then a car — you don't want to build a car first, badly. The ability to quickly prototype and test ideas with users is still core to the role; the iteration speed is just different. In theory, all three roles of the product trio — the designer, the PM, the engineer — the lines are blurring, and I don't necessarily know what the implications of that are. The piece of the puzzle where I'd previously ask an engineer to spend two weeks on a prototype, then explain the gaps they didn't understand, then try to turn it into a product vision and test it with users — I can in some ways do that in an hour now. I can understand the problem, test an idea before I even ask an engineer to do something, even come up with a working prototype. Is that useful? I'm not sure. We're still learning how to manage these superpowers. There's also a real spectrum of bullishness across all these roles. Even experienced people who use Claude for a lot of things aren't necessarily fully bought in — there's some trepidation in different places, some of it justified, some of it just scepticism. And there are obvious questions about how mentorship works when you didn't learn the fundamentals. Every generation of computing has said this — the assembly people told the high-level-language people, "you won't understand how this works." This time feels different, but it's also familiar. So the role is hard to know how it evolves. I feel like I've got superpowers; I don't always know when to wield them.

James Mulligan05:24

What do you mean by that?

Joel Schwarzmann05:26

Let me transition into it. Marty Cagan recently published an evolution of his thinking, separating "build to learn" from "build to ship" — where an engineer's responsibility is the built thing, which means it has to be robust, it can't fall over, it's production-grade. If I'm building to learn, I don't have that requirement yet. I'm not telling my engineers "ship this." What I am doing is understanding: this is what the pathway looks like, this is where the failure modes are, this is the direction if we strip it down to its core principles, and these are the things we're discarding and trading off. What I'm describing is really just the double diamond — fan out, fan in. The ability to build something relatively high-fidelity, strip it down to the fundamentals, then iterate again and again — that's an interesting way of working that would once have taken multiple people and months to reach the fidelity I can now get to quickly. But if I'm self-critical: how do you avoid navel-gazing? How do you know it's not just "of course I like this, I built it"? How do you keep that user-centricity and grounding in reality? Still, I'm able to maintain flow states in a way I didn't used to.

James Mulligan07:03

That flow state is becoming a bit of an addiction state, isn't it? I've felt that myself. So the Cagan framing was build-to-learn versus build-to-ship — build-to-learn being discovery: you're building something to learn from users whether you're building the right thing, but it doesn't need to enter production yet. And you're saying that part of the process has been accelerated — crushed into a much shorter period that requires fewer people.

Joel Schwarzmann07:31

Yeah. In the Cagan style — design-led, research, empowered teams, clear vision, clear requirements — understanding that discovery lifecycle, as well as the soft side of the role: motivation, bringing people on the journey, building conviction, managing stakeholders. Being able to have a living, breathing prototype I can keep showing people — "this is the vision, this is the way it should work, this is the evidence for why" — feels like a powerful thing I can't step away from. And it's true in the coding editor, but even in tools like Notion I'm using the agent sidebar most of the time. I'm very happy doing that same build-up, strip-down, iterate journey — whether I'm writing a document or writing an application, it's the same thing.

James Mulligan08:46

And you said earlier that the underlying best practices — how you write a PRD, how you think about user stories — are still important, because that's what you're using to editorialise your prompts to the agent.

Joel Schwarzmann09:06

Yeah. I think it's still true that you can't do any of this waterfall — you can't possibly know all your requirements, all your failure cases, everything users will actually experience, by writing a really detailed spec up front. But the classic primitives — user story, functional requirements, acceptance criteria — are all still really useful, and agents follow them. You can even ask an agent, "do you agree we've met the acceptance criteria?" You don't have to trust it, but the technique is available to you. So the craft hasn't gone away. There are areas where I feel more productive — where I can understand a problem and build conviction, at least in myself, that we should go in a particular direction, by having an all-knowing assistant at all times. Closing that loop into a revenue-generating product — I've not got that far. I have high conviction we'll get there. And the operational side — a running, living, breathing system — is an interesting space where agents could be on call later on, too.

James Mulligan10:44

So you're saying this "build to earn" piece hasn't quite been figured out yet.

Joel Schwarzmann10:48

At least I personally haven't completed the loop of shipping something that's genuinely revenue-making. I'll get there — but not yet.

James Mulligan10:59

What do you think the gap is? What means a human still needs to be responsible for the build-to-earn piece?

Joel Schwarzmann11:07

It's the gut feeling and pattern-matching of what it takes to understand how a system works. Now, that could be documented really well — specified in markdown files, schemas, skills, all of that. But I don't think the rough edges, the skeletons in that part of the product, are really known. You still need someone with deep understanding of why the system works the way it does to really drive things. That's different from saying, "I want to ship this feature."

James Mulligan11:50

So you're describing accountability.

Joel Schwarzmann11:51

Accountability, knowledge of the moving parts, and prioritisation of the moving parts. A simple, almost philosophical question: does an agent understand time? It can be trained to say "this is probably a week's worth of work" — but does it feel any pain in suggesting a week's work versus a day's?

James Mulligan12:16

That's so interesting. I've seen it come back with estimates — "this'll take someone a few hours" — and it's like, no, you're going to do it, it'll take you 30 seconds.

Joel Schwarzmann12:25

True. I still think a human responsible for a production system understands the different dynamics — the downtime costs, the teams that need alignment, the stakeholder management. And I'm pretty bullish this will be figured out, that more and more can be driven that way. But I think it's naive to say we're there. I'm also in two minds about whether we're Uber in 2007 — where it's five dollars, entirely VC-subsidised, and driverless cars feel right around the corner, but 15 years later we're still on the long tail of making that real. Even Waymo, as cool as it is, isn't cheaper than a taxi in any way, shape or form. That's the cynic in me — we've reached a really sophisticated point, but where's the next frontier, where's the next leap coming from? Or are we still on that exponential curve? Then again, five years ago ChatGPT was still two years away — which is a bonkers statement, given how much is different now in every way we work.

James Mulligan13:49

It definitely feels accelerated — tantalisingly so. Taking a step back from how a product team operates and how that shapes your workflow — I've heard you talk a lot about the concepts underneath building an AI system: the harness, and so on. These are emerging topics people weren't even discussing a year ago. Can you talk a bit about that?

Joel Schwarzmann14:17

The vocabulary we have is all changing. We're also in the hype cycle to end all hype cycles, and there's a lot of "oh, you're into that? I was into it six months ago" energy.

James Mulligan14:37

Right — like, last year it was all MCP, now it's [the next thing].

Joel Schwarzmann14:42

As cool as [open-source agent — name to confirm] is, I've seen far fewer people talk about it in the last month than how world-changing it seemed four months ago — in the same way the debate between CLIs and MCPs was a cool-kids position a couple of months ago. But I'd also point out that the GitHub CLI is the best out there, and their MCP server is the world's most popular MCP server. Perhaps there's a world where they coexist for different problems. So all these terms are emerging. What's also fun to watch is the whole world throwing every idea at the wall for the last three years — and some of the good practices are shaking out. Tool calling, for instance, was something every team was doing individually; then Anthropic pushed the MCP standard, lots of excitement, lots of adoption, and the limitations kept being ironed out. Then the emergence of skills, the emergence of [CLAUDE.md — to confirm] as the right place — and the fact that all three of those primitives turned out to be pretty complementary. There are more ambitious, longer-term things — like A2A as a protocol — that I see people adopting in statements, but I haven't necessarily seen the value yet.

Joel Schwarzmann16:01

It's also interesting from a business point of view: where's the competitive advantage? There are a lot of people doing tracing and evaluation platforms. Is there a business there, or do you just get commoditised by open source or by the cloud players? And it's interesting to see my old world — the orchestrators, things like Dagster, Prefect and Airflow — which are ostensibly related to this space but aren't necessarily best placed in this AI cycle. Prefect's pivoting, maybe. The point is, the tools that are useful here are still being defined. There's a lot of activity, and a lot that'll be consolidated in the next couple of years. It's really hard to know where the companies are going to thrive — whether it's going to be on the vertical or the horizontal, or whether everything gets swallowed up by Anthropic and OpenAI as the model providers. It's a super-interesting time.

James Mulligan17:17

Let's talk predictions. There's a lot of chat, mainly on LinkedIn — big organisations letting engineers go and blaming AI, when sometimes there isn't actually AI behind the scenes; surveys of Fortune 500 CEOs who probably aren't using AI themselves commenting on the future of work. You work at an AI company, you're building agents — in a year's time, how do you see an organisation being structurally different? Same roles? Do we build products the same way? What's likely?

Joel Schwarzmann18:00

I don't think we'll build products the same way.

James Mulligan18:03

Because you have evidence of that today.

Joel Schwarzmann18:09

Yeah. If you think of the discovery loop — a meeting recording, a Linear ticket, the ability to implement something — I think that can change how things are staged for contribution and reviewed, and where the senior engineer provides their steer. I'm kind of assuming no one's writing the code anymore, but they're still responsible for it. The two are different. There's a fork in the road on predictions: are we going to see jumps like we've seen before? I'd argue this GPT-5 class is cool, but it hasn't felt like the jump from GPT-3 to GPT-4 did originally. There's also the emergence of open-source and Chinese models — how much that drives down unit costs and competitive advantage, and how much the harness layer gets commoditised. You see things like managed agents from Anthropic, which is an interesting play: "we'll do it all for you — you're already paying us for some of it."

James Mulligan19:34

So you see harnesses as something not owned by the software vendor, but becoming an orchestration piece for the bigger AI players that people stick on top of their product.

Joel Schwarzmann19:46

Like anything, there are times you opt for best-in-class and roll it yourself, and times it doesn't make sense to. That's always been true.

James Mulligan19:58

Would you still train to be a product manager today?

Joel Schwarzmann20:02

I just changed my title to builder.

James Mulligan20:06

You changed your title to builder — elaborate on that. I think I know what you mean, but what do you mean?

Joel Schwarzmann20:14

It's related to the wider conversation about CTOs becoming "member of technical staff." I think the spikes of design, product and engineering are still valid — but the roles will blur. I know, slightly unfairly, that I'm probably better at stakeholder management than some of my best engineers — that's just a different personality spike. But I also feel I have the skills to be the right translator between those roles, and there's always a place for that. You're still going to need to get people to agree on decisions, alignment and direction. So the role's going to be very different. And the question of what the next generation leaving university does is a very valid and interesting one.

Joel Schwarzmann21:13

I don't know if I'm in the cynic or the optimist camp. The obvious cynic position is that they don't have the apprenticeship of what it took to write code by hand, so they don't know where they're going wrong. The optimist argument is: if you're born and bred with that tooling and that superpower, how much more productive can you be, unencumbered by any of the old questions? It's the classic debate about allowing a calculator into an exam — what do you actually need to understand?

James Mulligan21:56

So you're arguing that the predefined roles of the past two decades might not be so well-defined going forward — but the skills are still needed, the craft is still needed, and an understanding of those things is still needed to be a product builder.

Joel Schwarzmann22:09

I think so. And it depends on the kind of product, the kind of organisation, the kind of team. I also think we'll see a lot of smaller organisations — the revenue-per-employee number will be really interesting. There's that question of when we see the first one-person unicorn. I read about someone running an agent that looked at satellite imagery, then called local pool-installation companies and took a commission on the leads it generated. Obviously there aren't an unlimited number of ideas like that, but for effort versus reward, that kind of thing is very interesting.

James Mulligan23:01

Fascinating. Well — maybe we should make a list of those ideas, Joel, and see whether we can make them real. Look, always a pleasure. Thank you very much for your predictions. We'll circle back in a year's time and see which came true.

Joel Schwarzmann23:16

Thank you.

James Mulligan23:18

Farewell. Right — done. Legends, thank you very much.