We're training a Figma-fluent editing model
94 points - last Sunday at 4:05 AM
SourceComments
For context, this blog was released the same day Canva made their announcement for Brush, which looks to be further along in this area: https://www.canva.com/newsroom/news/canva-brush/
I'm glad Figma is pushing in this area. I do think that Figma has one of the best UI datasets out there in the world, and for the most part they've been sitting on it letting it get unused. Excited to see where they take this, but I do think they should be looking at image models in addition to LLM-based edit models (I'm biased as I left figma to start a image-mode-based UI design tool) as I find they're more capable in this area.
In my experience, Figma Agent and Figma's MCP leave something to be desired. I'm sure Figma Agent is useful for people who haven't used Claude Code or similar before, but it really requires most of your product's context to exist in Figma for it to do well.
As soon as your product's context exists out of Figma, then it starts to drift and hallucinate its own styles. Using the MCP with an AI that has access to your code provides better results, but even it will have trouble remembering what page I'm asking it to edit.
I know they're trying to address this by adding features such as guidelines and specs, along with linking your Figma files to GitHub repos, but my complaint with these features is a lot of the companies I have worked for do not use the very specific tech stack that Figma supports with these features. If it's anything like code connect, it may take years to get more broad support. Every company I have worked for is a C# Microsoft/Azure house.
This does not help with my broader frustration with Figma which is there is a lack of focus/vision. They're trying to compete on all fronts by developing all these different products/features and it doesn't really feel like they have a cohesive story for what the future or product design is. Granted, I think most of us in this industry don't either, but when am I supposed to use Figma Make vs. Figma Agent vs. Code layers?
They keep blurring the line between them when in all honesty I'm more interested in just using the AI agent I pay for to code what I need and bring it into a canvas for archiving/exploration when I need it.
Feels to me like a big issue that a smart competitor could take advantage of. I want to use my agent, I don't want to pay Figma to use their agent.
I'm also a bit salty about Figma ratcheting down hard on API calls, which creates customer service issues for any product that integrates with the Figma API. So maybe I am biased, but based on comments in here it also seems like the Figma MCP isn't very good either. Is Figma's long-term strategy to go full walled garden?
I find it a huge distraction to ever draw anything in Figma now
Then the whole thing is effectively useless because of server-side font licensing. Your expensive brand font cannot be used by the MCP so you then need an additional layer of client-side Figma plugins to replace the placeholder font with your real font.
I know figma has a strong fanbase and companies are all rolling their own "post-trained" models for their built-in AI, and figma is a great product, but the fact frontier models can basically do in html what figma does in it's intermediary format sort of point to where the chips will land in terms of what you will need to use figma for in a few iterations.
I haven't used canva in 6+ months either for the same reason and I don't think either's userbases are growing honestly.
I know Dylan Field recently posted they made the Llama's 70B model do amazing things but the AI training field is an absolute snake pit.
But then again they have money to burn ig.
Easier to maintain, scale, and can use most of the components of the design system directly if you're doing web.
Bitter lesson, etc, and training an entire VLM to prevent text overlap seems a bit much, especially since GPT 6.* /Claude 5.*'s vision are good enough that it rarely ever happens for me, and if they do make a mistake, it's generally a 5 minute fix?
Figma has consistently done better than most at nailing the UX. Maybe I'm enough of a generalist that it always feels great to me, and super specialists lament the lack of their super special needs being met (genuinely, a valid gripe for the specialists).
I've tinkered with Figma AI, and it's just good enough to keep me tinkering. It's not creating magic yet, and it's certainly not outpacing claude design to the point where I always pick figma.
I am enthusiastically optimistic that Figma will push the UX on agentic design to where I come to figma every day again.
Its surprising to me that the market is still stuck on SaaSpocalypse not noticing that these companies have pivoted hard in an impressive and revenue generating way
Its to the point where their consumer facing products are a sideshow and distraction, but thats what people are familiar
They’re selling tokens just line all AI forms, with huge enterprise commitment payments up front
2.) You can use any agent with Figma
3.) You can use any agent with not-Figma
I used to really respect Figma's engineering team and have been blown away by the degree to which they've dropped the ball with AI.
All they need is to expose their existing API very well over MCP and give a "screenshot" feature agents can utilize and the agents are good to go.