Using AI for UX, Designing and Building Prototypes

Artificial intelligence tools are changing the way we work. From design to development, the digital product development process is being turned on its head. 

Creating a prototype, the lean way

If you’re an app creator looking for feedback, funding, or just looking to sharpen your development skills then you’ve probably made your fair share of prototypes. Typically a prototype only contains core features, the parts of an experience that are unique to your product and are necessary to perform user testing. Depending on the fidelity you’re aiming for, you might not need unique branding or styling at this stage. Instead, you might rely on prebuilt interface components, like SwiftUI or Google’s Material Design libraries.

If you’re a little farther along in your product design and development journey then you might instead choose to build minimum marketable features. These are features that have measurable value, consumers or businesses would be willing to pay for them.

In short, you don’t have to build an entire, best-in-class, or ready-for-market product. You can build out the most important pieces, get feedback, improve your design, and work on securing funding or continuing to bootstrap your app. Building a high-quality prototype ultimately means you could bring your ideas to market and they would meet user's needs, while still being lean.

Using AI to reduce prototyping costs

Creating a viable digital product takes a lot of interdisciplinary skills: research, brainstorming, planning, design, development, QA, and marketing all play a role. It’s hard for one person to truly wear that many hats. Sure, you might be a great General Manager but can you also design a top-notch UI or ideate and code complex functions in multiple programming languages?

...You shouldn’t try to use AI to create a finished product. Instead, use it as a virtual team mate that’s there when you need it.

This is where AI can help, you can fill in some of your team's skill gaps using tools like large language models or generative AI. If you’re looking for help brainstorming ideas you might use a tool like ChatGPT or Gemini. Maybe you need hero art or graphics? You could use tools like Adobe Firefly, Midjourney, or DALL•E. If you’re looking for ways to speed up production you might consider Cursor AI or GitHub Copilot for your development needs or Galileo AI to help generate basic Figma files.

Can AI build a prototype by itself?

Anyone can leverage AI tools, but that doesn’t ensure your end product is going to be functional, valuable, or successful. You still need to have some baseline knowledge of the disciplines you’re working in. Think of AI as a team member who’s there to help extend your creative, technical, or analytic skills, and not as a replacement for critical thinking skills. You shouldn’t try to use AI to create a finished product. Instead, use it as a virtual teammate that’s there when you need it.

Using AI for UX & UI design

The number of AI tools made for UX & UI designers is growing every day, it’s next-to-impossible to catalog them all, as soon as you do another is likely to pop up. Some common use cases for these tools include:

• Writing requirements for common features: This works great for experiences that are similar across digital products, like subscribing to a service, adding an item to a cart, or logging into an app.
• Creating user personas and flows: You can use an LLM to help flesh out existing personas, use Figjam’s built-in AI tools to create flows quickly, or leverage plugins to do both. Figma’s plugin gallery has a few options you can test-drive too.
• Generating interfaces, icons, color palettes, or hero art: You can leverage some of the tools we listed earlier or try asking an LLM to generate requirements and CSS for basic states: active, inactive, hover, and click. You might be surprised at how comprehensive the results are.
• Automating repetitive tasks: Creating variations for a list interface, making mocks for user profiles, or inserting placeholder text or images. Automator can take of most of these, and there are a ton of custom flows built by community members too.

Making a flow using FigJam AI, a case study

Again, you’ll want to use caution whenever you’re using AI tools—they’re not a replacement for knowledge or critical thinking skills. Looking at the chart above, FigJam’s AI was able to create a pretty basic login flow that includes signing in using an email or via Google or Apple. We used the following prompt,

…if an AI has seemingly met your requirements, then it’ll recieve positve feedback regardless of wether it generated the “best” solve.

FigJam’s Ai wasn’t wasn’t the most efficient. For example, the AI could have grouped all third-party logins into one step, as those interactions are managed by their owners, and then listed 3P login services either in a footnote or in some other documentation. This would have simplified the chart, creating a less busy and easier-to-understand flow.

AI doesn’t necessarily care (can it care?) about efficiency, simplicity, or best practices. Reward modeling creates a system where any given AI is more worried about meeting requirements than about overall quality. Meaning, if an AI has seemingly met your requirements, then it’ll receive positive feedback regardless of whether it generated the “best” solve. This sometimes creates unintended solutions. Knowing this, you can create prompts that are specific to your problems, clearly lay out requirements, and are concise. Following these best practices should give you more successful AI generations in general.

Using AI for developers

Over the last few years, there has been a shift towards software developers and engineers using AI tools to assist in writing, editing, and optimizing code. One of the most common tools is Copilot, developed by Github, and made to be used with your editing program of choice—extending functionality, boosting efficiency, and bringing natural language chat to the party.

There are also code editors that use specially trained LLMs as a core offering, chief among these being Cursor. Built as a fork to VS Code, Cursor AI’s native implementation means you can highlight sections of code and ask for help understanding context within a codebase, make updates and include inline comments to help understand changes or refactor code to work with different frameworks or libraries.

Talking with Lowell Pence, a Software Engineer at Rapptr, he pointed out how useful Cursor can be when working with unfamiliar systems,

”…getting setup on a new project, and working through setup errors…tool is super helpful. Especially when there are multiple errors, and you aren't familiar with the tech stack.”

Further, Cursor can provide “thoughts” or summaries of issues, walk you through new tech stacks, and propose solutions to errors or bugs. Like most AIs, it isn’t accurate 100% of the time, but is still a very useful tool for quickly gaining understanding or making updates.

Will AI replace programmers?

Simple answer: No, AI will not replace programmers. At least not all of them, not fully, and definitely not yet. Programming requires critical thinking skills that our current LLM AIs don’t or can’t possess. Until AI can understand and solve problems developers don’t need to worry about being replaced.

Building with Vue.js using AI, Figma and Cursor

To demonstrate a few features of the tools covered in this article I put together a JSFiddle. The example uses HTML and CSS generated by Figma, based on an AI-generated layout from Galileo. I then asked Cursor to refactor the HTML as a Vue app, including a few fundamentals like data-binding for the button text and methods to log a string to the browser console to make sure everything was working.

If you’d like to recreate this case study, make sure you have both Figma and Cursor installed before starting, and then follow these steps:

1. Create a prompt: Use an LLM AI to generate a UI prompt for Galileo. In this case, we create a prompt that puts two buttons side-by-side, but you can experiment with more complex views too.
2. Generate Figma files using Galielo: Copy-paste your prompt into Galileo, hit “generate” and wait for the magic to happen. Then, copy-paste the resulting files into Figma and make sure your layers are all set up using auto layout a.k.a. CSS Flexbox.
3. Copy HTML from Figma to Cursor: Once your files are set up, switch your Figma to Dev Mode and navigate to the “Code” section in the right sidebar. Update the dropdown to “HTML”, select the component you’re converting to code, then copy the HTML that appears in the “Code” section below the Box Model. Last, add your copied code to a new Cursor file.
4. Refactor your HTML as Vue.js: The HTML that Figma provides includes inline CSS. First, let’s move that CSS into a <style> tag. Then, we’ll ask Cursor to refactor everything.
   1. Figma typically puts each graphic element into its own <div>. Highlight all of your code, and then press cmd-k to start Cursor’s chat. Prompt Cursor to move the inline CSS to a <style> tag. Generate results and press “Accept” if they look correct. You might also consider making semantic updates, i.e. "Replace all <div class="button"> with <button> tags".
   2. Finally, highlight all of your code, press cmd+k, and ask Cursor to refactor everything as a Vue.js app. You can preview this in a browser or move it to a JSFiddle to share and fork with others.
5. Add extra features, optional: To push this test a little further we asked Cursor to add the following: hover states, data-bindings for the button text, and methods that log “button pressed” success messages to the console. 

To keep things simple, I created the entire project in a single HTML file. You could extend this idea though, to build: Vue templates, React components, or other reusable snippets. Combining AI, UX, and software development has never been easier—the sky’s the limit.