Why I Decided Not to Automate Concept Development with AI

I built an AI agent team to test concept development. The results were useful. But building the process into an AI product made one limitation clearer: useful AI outputs are not the same as the human alignment required for concept development.

I created a cohort of AI agents to act as a cross-functional team for concept development. I started the project to learn more about agents and bots. By using agents, I was able to isolate process variables that are difficult to isolate when real people are working together to develop a product. 

The purpose: test concept development processes. AI made it possible to do it without proprietary company data or sensitive information. Could the same exact team (fresh for each case, starting from the same point in the project with the same background knowledge) produce better concept-development outputs when using a different process?  

The Four-Case Concept-Development Experiment

I did four concept development cases to test out different limits, assumptions, and challenges.  

My baseline: I defined a typical concept development process. 

My delta: I taught the AI agents the Concept Space Model and the ADEPT Team Framework.  

This was not a substitute for a live product-development team, and it was not a statistically controlled study. It was a way to isolate the process variables and make the differences visible. 

Examining the results, the difference between the baseline and delta was clear. The team of agents that used the Concept Space Model and the ADEPT Team Framework consistently produced more clearly defined engineering inputs, better connections to the end user and product, and action items that drove the project forward.

That’s what we want concept development to do: reduce ambiguity, expose assumptions, define a path forward, link it to the business and the end user. 

You can see the results here: Your Concept Development Isn’t Done Just Because It Produced Answers – Deeney Enterprises

Testing AI to Automate Concept Development

Then someone suggested I should develop it into an AI product. And I started to. I even created an account on GitHub and used Claude Code. I included human-in-the-loop controls. But the more I worked on it, the clearer the limitation became: I did not want to build an autonomous replacement for the team. 

My experiments demonstrated that an AI agent team could apply the methods in Pierce the Design Fog and produce useful concept-development outputs. The Concept Space Model and ADEPT Team Framework were effective at aligning the agents around the work. Together, they could surface assumptions, develop engineering inputs, and organize the work. 

The disconnect was not in the methods. They worked. The disconnect was in the team.

AI agents can simulate cross-functional perspectives, but they are not the people whose judgment, tradeoffs, and commitment will shape the product. What was missing was the human alignment and accountability those decisions require. 

Why AI Agents Cannot Replace the Team

Using AI agents to complete concept development on their own can recreate the same ‘throw it over the wall’ pattern that collaborative concept development is supposed to prevent. The handoff has simply moved from one department to an AI system. We want to eliminate that pattern, not introduce a new way to do it.  

This is related to the ‘decision room carry’ problem: a decision does not automatically carry into the work simply because it was made and documented in a room-or generated and documented by an AI system. 

My Conclusion: Use AI to Support the Team

Use AI to make the team better informed and more effective. Use it to support the team or facilitator for preparation, reinforcement, or a first pass.

Don’t use it as a substitute for the teamwork needed in concept development. Don’t automate concept development by removing the team from the work.

This is also why I work with product and engineering teams on the front end of development: not to add another tool, but to improve the quality of the conversation before decisions become expensive. 

If your product or engineering team is exploring AI in the front end of development, I’d be glad to compare notes.  Contact me..