.png)
2 hours ago
1
This week’s newsletter is sponsored by DX.
How 18 Top Companies Measure AI’s Impact on Engineering
Written by Laura Tacho, CTO at DX, this new industry report explores how 18 engineering organizations—including Dropbox, Microsoft, GitHub, Atlassian, Adyen, Booking.com, and Grammarly—are measuring AI’s impact on software development.
Inside this report, you’ll learn:
Common patterns: Why most companies track speed, quality, and maintainability alongside AI usage.
Unique metrics: A deep dive on Microsoft’s “Bad Developer Days” and Glassdoor’s experimentation measures.
Practical guidance: How to get started with the AI Measurement Framework and guidance for collecting metrics.
Thanks to DX for sponsoring this newsletter, let’s get back to this week’s thought!
Using different tools that are available to make your day-to-day easier and more productive is what great engineers do.
AI is a tool like any other just the difference is that it’s a lot more ambiguous to use. You rarely get a “Complete Guide” on how it can actually be helpful, specific to your daily tasks.
But in this article, we’ll try to get as close as possible to give you a complete blueprint!
There are a bit more code examples than in a regular article, and the reason for that is that they are extremely useful and helpful. They give you a blueprint on how you can structure a specific prompt and what output you can expect.
You can use this framework for a variety of different SWE tasks. We’ll go through examples of what is possible. Definitely try it out and let me know how it goes!
If you like these kinds of articles, where we share insights on how you can use AI to be more effective, you’ll like these articles as well:
Let’s introduce our guest author for today’s article and get started!
Steven Levey is Founder and CEO at Revtelligent, with over 15 years of experience leading innovation across startups and large enterprises.
He’s built and scaled multiple tech companies from the ground up and led transformative AI initiatives at public companies like SailPoint.
Today, Steven specializes in helping organizations move from AI theory to real-world impact → translating hype into measurable business outcomes.
At Revtelligent, he’s focused on helping engineers and leaders to become indispensable AI authorities within their companies.
This framework provides a structured, repeatable method for constructing high-quality prompts to get the best possible results from AI models for software engineering tasks.
It builds on the core concepts of providing persona, context, and clear instructions, but organizes them into a logical workflow that mirrors how a developer might approach a problem.
The acronym C.R.A.F.T.E.D. stands for:
Context: Provide the background and the “what.”
Role: Define the persona the AI should adopt.
Action: State the primary, specific task.
Format: Specify the structure of the desired output.
Tone: Define the communication style.
Examples: Show, don’t just tell.
Definition of Done (Constraints): Set the rules and boundaries.
Context: Provide the background and the “what.”
Goal: Prime the AI with all the necessary information before giving it a task. This is the most critical step. Just as you would read a bug ticket or user story before writing code, you must provide the AI with the relevant “books from the library.”
Why this is important: Placing the context first “primes” the model, ensuring its entire “thought process” is grounded in the specifics of your situation. It prevents the AI from jumping to conclusions based on a premature instruction and forces it to analyze the provided data before acting.
What to Include:
Code Snippets: The function, class, or component to be worked on.
Error Messages & Stack Traces: The full output of a failed process.
API Documentation or Schemas: The structure of data or endpoints.
File Diffs: The changes from a commit or pull request.
User Stories or Requirements: The description of the feature or bug.
Relevant Configuration: package.json, Dockerfile, tsconfig.json, etc.
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> I’m working with this JavaScript utility function: ```javascript // utils.js export function isString(value) { return typeof value === ‘string’; } ``` I also have this other function in a different file that needs to be updated: ```javascript // dataProcessor.js import { isString } from ‘./utils.js’; function processData(data) { // ... some logic if (typeof data.name == ‘string’) { // This line is non-compliant console.log(”Processing a string name”); } // ... more logic } ``` Our team’s style guide explicitly states: “For all runtime type checking, wrapper utility functions from `utils.js` must be used instead of the `typeof` operator directly. This ensures consistency and allows for easier logging and debugging in the future.” </context> Update the `processData` function in `dataProcessor.js` to comply with the provided style guide rule. </prompt>Resulting AI Output:
Role: Define the persona the AI should adopt.
Goal: Tell the AI who it should be. This focuses the model on a specific domain of knowledge, improving the relevance and quality of its response.
Why this is important: Defining a role provides a lens through which the AI should analyze the context. An instruction to “find issues” will yield vastly different results if the AI’s role is a “cybersecurity expert,” a “performance optimization specialist,” or a “UI/UX designer.” It narrows the model’s focus to the most relevant part of its training data.
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> Review the following AWS IAM Policy document. This policy is intended to grant read-only access to a specific S3 bucket for an application. ```json { “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Action”: “s3:*”, “Resource”: “arn:aws:s3:::my-company-reports” } ] } ``` </context> <role> Act as a principal security engineer specializing in cloud infrastructure. Your primary concern is identifying potential vulnerabilities, race conditions, and insecure defaults. </role> Analyze the provided IAM policy for security vulnerabilities. Identify any overly permissive settings and explain the potential risks. Provide your analysis in Markdown. Use a heading “## Vulnerability Analysis” and then a bulleted list to describe each issue. </prompt>Resulting AI Output:
Action: State the primary, specific task.
Goal: State the single, most important task you want the AI to perform. Be specific and unambiguous. Use strong, direct action verbs.
Why this is important: A clear, direct action verb removes all ambiguity about the primary objective. After being primed with context and role, the AI is ready for a specific command. Vague requests lead to generic answers; a precise action leads to a precise result.
Example:
“Generate three edge-case unit tests for the provided Python function using the pytest framework.”
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> Here is a Python function that calculates the sum of squares for a list of numbers. ```python def sum_of_squares(numbers): return sum(x*x for x in numbers) ``` </context> <role> You are a software engineer specializing in test-driven development with Python. </role> <action> Generate three distinct unit tests for the provided Python function using the `pytest` framework. Each test must cover a different edge case: an empty list, a list with negative numbers, and a list containing zero. </action> Return only the Python code for the tests in a single code block. Do not include any explanation. </prompt>Resulting AI Output:
Format: Specify the structure of the desired output.
Goal: Dictate the exact structure of the output. Never assume the AI will guess the format you want. Being explicit here saves significant time on re-rolling or editing.
Why this is important: Specifying the format makes the output immediately useful. Without explicit instructions, you might get a perfect code snippet buried in conversational text. By defining the output structure (e.g., “return only the code,” “use a JSON object”), you get a response that can be immediately copied, pasted, and used programmatically.
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> Analyze the following Python function: ```python def calculate_average(numbers): if not numbers: return 0 return sum(numbers) / len(numbers) ``` </context> <role> You are an automated code analysis tool. </role> <action> Analyze the function and provide its name, a list of its parameters, and its return type. </action> <format> Return the analysis as a single, minified JSON object. The object must have three keys: ‘functionName’ (string), ‘parameters’ (an array of strings), and ‘returnType’ (a string, e.g., ‘number’, ‘string’, ‘any’). Do not include any other text or explanation. </format> </prompt>Resulting AI Output:
Tone: Define the communication style.
Goal: Specify the communication style. This is crucial for tasks like generating documentation, code comments, or explanations for different audiences.
Why this is important: Tone shapes the output for its intended audience. A code comment should be concise and technical. An explanation for a non-technical stakeholder should be simple and high-level. A tutorial for a junior developer should be encouraging. Defining the tone ensures the communication is as effective as the code.
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> The junior developer is confused by this line of Python code: `results = [process(item) for item in data if item is not None]` </context> <role> You are a patient and knowledgeable senior developer pair-programming with a junior developer. </role> <action> Explain what this line of code does. </action> <tone> Adopt a friendly, encouraging, and slightly Socratic tone. Ask questions to guide the user toward the answer before providing it. Don’t just give the answer directly. </tone> </prompt>Resulting AI Output:
Examples: Show, don’t just tell.
Goal: Provide a crystal-clear demonstration of the desired input/output pattern. This technique, known as “few-shot prompting,” is the single most effective way to get precise, custom-formatted results.
Why this is important: An example is the ultimate form of clarification. It removes all ambiguity from your request, especially for complex or custom formats. It’s the difference between describing a blueprint and showing the AI the blueprint. For non-standard tasks, this is the most powerful tool to guarantee a correct response.
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> “I fixed the login button and also refactored the auth module.” </context> <role> You are an expert at writing Conventional Commit messages. </role> <action> I need to convert a simple, natural language description of a code change into a structured Conventional Commit message. </action> <examples> Here is an example of the required transformation: Input: “added a new feature for user profiles” Output: “feat: Implement user profile feature” </examples> Return only the final commit message as a single string. Do not include “Output:” or any other labels. </prompt>Resulting AI Output:
Definition of Done (Constraints): Set the rules and boundaries.
Goal: Set the final constraints, boundaries, and success criteria. This is where you state what the AI should not do and any other critical requirements.
Why this is important: Placing rules at the end leverages a phenomenon known as the “recency effect” → models tend to pay very close attention to the last instructions they receive. These final constraints act as a hard filter on the output, ensuring the AI adheres to your most critical boundaries (e.g., “do not use external libraries”) as it generates the response.
Working Example: Using C.R.A.F.T.E.D
<prompt> <context> I have a nested list in Python that I need to flatten into a single list. ```python nested_list = [[1, 2, 3], [4, 5], [6]] ``` </context> <role> You are a helpful Python programming assistant. </role> <action> Write a Python function called `flatten_list` that takes a nested list and returns a single, flattened list. </action> <format> Return only the complete Python function in a code block. Provide no explanation or surrounding text. </format> <definition_of_done> Constraint: Do not use any external libraries like numpy or itertools. The solution must be implemented using only standard Python language features. </definition_of_done> </prompt>Resulting AI Output:
Try the above, without the definition_of_done, and you will get different models using 3rd party libraries instead of the built-in libraries.
Scenario: You have a slow Python function and need to optimize it and get an explanation for a code review.
C.R.A.F.T.E.D. Prompt:
<prompt> <context> You are given the following Ruby method which is inefficient for large arrays: ```ruby def find_common_elements(list1, list2) common = [] list1.each do |item1| list2.each do |item2| if item1 == item2 common << item1 end end end common end ``` </context> <role> Act as a senior Ruby developer who is an expert in writing clean, efficient, and idiomatic code. </role> <action> Refactor the method above to be more performant, especially for large input arrays. </action> <format> Return your response as a Markdown file. Include the optimized Ruby code in a code block first. Following the code, add a section titled “## Explanation” with a bulleted list detailing the changes and why they improve performance. </format> <tone> The tone of the explanation should be clear and professional, suitable for a pull request description. </tone> <definition_of_done> - Do not use any external gems. - The method signature (name and arguments) must remain the same. - The solution should handle empty arrays as input without raising an error. </definition_of_done> </prompt>Yes, I know there is no <examples> section in the above. That’s on purpose! (See closing thoughts)
Resulting AI Output:
# Optimized find_common_elements Method ```ruby def find_common_elements(list1, list2) return [] if list1.empty? || list2.empty? list2_set = list2.to_set list1.select { |item| list2_set.include?(item) } end ``` ## Explanation - **Converted list2 to a Set**: The original nested loop approach has O(n×m) time complexity. By converting `list2` to a Set, lookups become O(1) average case instead of O(m), reducing overall complexity to O(n+m). - **Used select with include?**: Replaced the explicit loop and array building with Ruby’s idiomatic `select` method combined with Set’s `include?` method, making the code more readable and leveraging optimized native implementations. - **Added early return for empty arrays**: Handles edge cases gracefully by returning an empty array immediately if either input is empty, avoiding unnecessary processing. - **Eliminated duplicate results**: The original implementation would add the same element multiple times if it appeared multiple times in `list1`. The new approach naturally handles this by checking membership rather than iterating through all elements. - **Reduced memory allocations**: Instead of repeatedly appending to an array with `<<`, we use `select` which creates the result array in a single operation, reducing garbage collection overhead. The refactored method maintains the same interface while providing significant performance improvements, especially noticeable with large input arrays where the difference between O(n×m) and O(n+m) complexity becomes substantial.Nearly all of the examples above are simple and are there to reinforce the ideas behind the framework.
In your day-to-day use with AI, if you truly have one-off basic questions or refactors, then you would not need to implement the full C.R.A.F.T.E.D framework. Use your discretion. Sometimes, you don’t have to add examples, tone, or format. Especially for the simple exercises.
However, when you start getting poor results, go back to this and layer in the sections until you get the results you need. This will also become a LOT more important and useful as code bases grow in complexity and the <context> sections start to get large.
Keep in mind that at the end of the day, it’s ALL CONTEXT. Each section, although wrapped in an XML tag for structure, is just context that is being provided to the model. You can manually build up the context, or you can programmatically build up the context (Tools; MCP; Webhooks; Data Pipelines, etc.)
Special thanks to Steven for sharing a LOT of value and a great framework that we can all use in our day-to-day engineering tasks!
Make sure to check him out on LinkedIn and also check out Revtelligent → he’s doing a lot of great stuff there.
We are not over yet!
After a lot of great feedback on Wednesday’s article: How to Build Trust as an Engineering Leader Part 2: Structure and Execute the Plan, I did a deeper dive on the most common problems new engineering leaders face and how to solve them.
New video every Sunday. Subscribe to not miss it here:
Liked this article? Make sure to 💙 click the like button.
Feedback or addition? Make sure to 💬 comment.
Know someone that would find this helpful? Make sure to 🔁 share this post.
Join the Cohort course Senior Engineer to Lead: Grow and thrive in the role here.
Interested in sponsoring this newsletter? Check the sponsorship options here.
Take a look at the cool swag in the Engineering Leadership Store here.
Want to work with me? You can see all the options here.
You can find me on LinkedIn, X, YouTube, Bluesky, Instagram or Threads.
If you wish to make a request on particular topic you would like to read, you can send me an email to [email protected].
This newsletter is funded by paid subscriptions from readers like yourself.
If you aren’t already, consider becoming a paid subscriber to receive the full experience!
Check the benefits of the paid plan
You are more than welcome to find whatever interests you here and try it out in your particular case. Let me know how it went! Topics are normally about all things engineering related, leadership, management, developing scalable products, building teams etc.
