AI in the Real World with Marlene Mhangami and Tim Allen
Published November 26, 2025
This video features Marlene Mhangami at DjangoCon Europe 2026 in Athens, Greece.
KEYNOTE: A Practical Guide To Agentic Coding For Django Developers by Marlene Mhangami
AI Agents have become increasingly good at generating code. Developers who know how to use agentic tools as they program can increase their productivity significantly. In this talk, Marlene will share how Django Developers can get the most out of coding agents in their development workflows. She'll walk through how she uses MCP (Model Context Protocol), Agent Skills and Instructions to create semi-autonomous agents that can complete multi-step tasks end-to-end. She'll also walk through best practices for validating the output of agents end to end with unit tests and Playwright. As part of this talk we'll also discuss the societal implications of AI on developers and how we can best prepare for the future thats both coming and already here.
More information: https://2026.djangocon.eu/speakers/MSY7X7/
Follow DjangoCon Europe on social media:
*X - https://x.com/DjangoConEurope/
*LinkedIn - https://www.linkedin.com/company/djangocon-europe/
*Mastodon - https://fosstodon.org/@djangoconeurope
#django #djangocon #djangoconeurope #djangoconeurope2026
Automatically transcribed, so expect mistakes in names and technical terms.
Speaker 1: Good morning. That was it. Thank you. Thank you. Okay, I see sleepy people, so let's do some hand waving. Who's been to Django Kon for the first time? No hands, I suppose, because you came yesterday. Yesterday was your first time. So welcome back. I hope you had a nice night last night. You rest a bit and then you're ready for the second day of the conference. Today, we have again full schedule. Bat we also have en nigter party with sam refreshments and drings efterk. So making sure that you stay here. Again, uh
Speaker 1: two floors down, it's the networking area plus you know uh coffee, lunch, etc. Everything's downstairs. The workshop room is also downstairs, it's not running right now, it's running I think at 11. 05 today. Um starting. And I think that's more or less it for now. Thanks again for being here. So let me introduce our first speaker for the day. It's uh Marlène Mangami. Uh she 's a senior developer advocate at Microsoft. She was the vice chair for the PSF and currently a PSF fellow. Uh she also led one of the first uh PyCon Africa and generally she's very active in the Python community And this is going to serve a
Speaker 1: practical guide to Atlantic coding. Uh I suppose that most, if not all of us have something to to do about this so please uh if you can have a round of applause for Marlène and welcome her today here. Marlène thanks for being here welcome
Speaker 2: Hi everyone, good morning. So my talk today is going to be a practical guide to agentic coding. uh specifically for Django developers, but actually we'll try to keep it quite broad. Just to note that the slides that I share from the talk today are going to be at the link that you can see on the screen So, a quick agenda of what we're going to be covering this morning. The first thing is we're going to look at the evolution of AI and coding. Then after that, we're going to cover coding agents and something called the agenc loop. And then I'll do some practical demos of actually using coding agents and how we can get the most out of them.
Speaker 2: And then finally we're going to look at some best practices as well if we have some time. So I think a good question for us to start with is what exactly is agentic coding? When we first started to see AI appear in our code, this was with something called code completion. So If you were writing a function in your IDE, maybe you would be typing and an LLM would look at the loss two or three sentences that you were writing and then it would create some suggestions. It would make some suggestions on which lines of code should come next. And then you could accept or reject those uh suggestions just like you do with autocomplete on your phone.
Speaker 2: After that, I would say about two years ago, we started to see the bigger labs like OpenAI and Anthropic actually start to create some LMs that were specifically designed for coding. And during this period of time, I remember going to ChatGPT maybe typing out some questions for a talk asking it to generate a script or just a function or something like that. And then I go back to my terminal, to my IDE, and then paste the code in, run the code, and then if there were errors, go back And paste it into ChatGPT, ask for help with the error, and then you know it would be sort of a loop. Who remembers that error? I don't know if you were experimenting during that time.
Speaker 2: But This was an interesting era because we really started to see that coding agents could get good, were really good at actually writing out code, but it was a lot of manual work copying and pasting in different places. And I would say about a year ago, so not very long ago, we really started to see agentic coding, true agentic coding workflows showing up. So for example, at Microsoft we released something called agent mode. And so if you are using uh copilot in VS Code and a user turns on agent mode, it turns copilot into this sort of autonomous AI peer programmer. So you could ask Copilot to complete a task on your behalf and then it would be able to on its own
Speaker 2: you know, execute commands in the terminal, be able to debug its own code and actually complete long running tasks in the background as well So this is when we really started to see um this sort of agentic coding taking off. And I would say the main thing that changed was that we started to give AI agents access not just to um just to uh you know uh a chat where it could give us feedback, but it had access now to our entire workspace so that it could also um take in some context and understand the context of our workspace and then use the tools that it had to take action. Today we have multiple different coding agents, and I'd actually say we're at a bit of an inflection point, which is really good for developers
Speaker 2: because There's a bit of a competition in terms of which coding agent is going to win out. So that means we have a lot of choice right now. So whether you want to use your IDE, you could use something like Copilot and BS Code. If you're in the CLI, some people like Claude Code. I would recommend copilot CLI instead, but it's up to you. A lot of people in the web use things like Codex or Jules. And then you can also create your own custom agents with tools like Pydantic AI or Langchain, for example, as well. So how many of you are actually trying out coding agents today and have tried out one of these? Great, that's fantastic. So we do have a lot of options and probably it looks like most of us in the room have tried them out.
Speaker 2: So I would say one of the best ways for us to understand how to get the most out of our coding ages is to understand how they work. So, even though we have this wide variety available to us, most of these coding agents work about the same under the hood. I think a good place for us to start in terms of understanding how coding agents work is to answer the question, what is an AI agent? And this topic has been kind of debated, but I think the industry has now come to terms with the definition on the screen there that an AI agent is an LLM that calls tools in a loop to achieve a goal. And I would say specifically for coding agents, they're associated with this agentic loop that you can see on the screen.
Speaker 2: It's a little bit fuzzy at the moment, but you can you can access the slides later if you want to take a closer look. And I actually got this image of or this specific Agentic loop from A blog post that the anthropic team uh put out called How Clawed Code Works Under the Hood. And for most of this talk, we're going to focus on the three uh parts, these three steps of this agentic. loop. So we're going to look at how to gather context, take action, verify results. So we're mainly going to focus using some live demos on how we can uh take advantage of our coding agents by doing the best we can at each one of these steps. So let's start with gathering context
Speaker 2: As a developer, the first thing that you are going to do if you're going to use a coding agent is you're going to send through a prompt. And when your coding agent receives that prompt, it's going to immediately start by gathering context. So I would say one of your top priorities should be to provide your coding agent with as much context as it possibly can. As you possibly can, and as quality context as you possibly can. There are a couple of different ways that you can provide context to your coding agent. So right now I'm actually in VS Code I'm using simple browser for my slides, but I also want to show you what we can do to add context. One of the most popular ways of adding context is to just click on and add your context directly to the chat.
Speaker 2: So here in Copilot, I could go ahead and choose one of these options. So I could attach a GitHub issue issue. When I'm using Copilot on a GitHub repo that's local, it will show me which issues and PRs are related to this um To uh the specific project I'm working on. So for example, here I've attached an issue and I can say, can you tell me about this issue? And then copilots should be able to get that information from that issue and tell me about that issue or even start to work with me to actually fix it. There's another a couple of other things you can do. You can also go ahead and directly attach files. One thing that I often hear from developers that are using coding agents is that
Speaker 2: Maybe they don't like the style of code that a coding agent is producing. And something I'd like to recommend is actually attaching a simple file with some guidelines or some example code to your chat. And having your agent use that as a starting point. So you're just attaching that file and saying always you make sure to follow the coding conviction conventions that you see in this file and then guide your agent using that. Another thing that we can use to give our agent context are instruction files. So for example, with If you're using GitHub Copilot, they're called copilot instructions. md. If you're using cursor, there'll be cursor rules, clawed,
Speaker 2: clawed. md. And something that's kind of gaining popularity in the industry right now is agents. md. And to show us an example of this, I'm going to actually just run this folder, this file, with a simple agent. And this agent uses It uses Copilot's SDK, which was recently released. And you can see here , I mentioned earlier that you don't necessarily have to use uh a pre-built coding agent like this one like Copilot here in VS Code you can also build your own. So for example with the SDK I'm gonna start a copilot session with that line of code and then when I create the session I can pick which model I want to use.
Speaker 2: I can send in tools And I can actually configure what sort of permissions I want to give the agent. So I'm just gonna pass through a prompt that says hello co-pilot. And then when I actually run the script I'm gonna run it just like I would any normal Python file. Oh, agent. py. When I run that , Again, the prompt that I passed through was just hello, co-pilot. It should give me a response. And it says, Hi, I'm Marlene's AI assistant for her DjangoCon Europe talk. How can I help? And this is interesting because when we look at the file, you'll see that there's no reference to Django Con. There's no reference to much in this file.
Speaker 2: And the reason why it's responding to me like this is because in my. github folder I have copilot instructions. That specifically tell the agent to behave in a certain way. So when a user re greets the agent, it's going to behave in this way. So when you're using things like copilot instructions, um or cursor rules, it's going to control at a repository level what your agent is going to do. So that's one thing way that we can add context. to your folder. I also mentioned that you can uh create agents. mds to create specific behaviors as well or complete specific tasks too. Something else I want to mention is context engineering, which is very popular these days.
Speaker 2: And when you're using coding agents, there was a study done by the team at Chroma, and they found that the more context that your coding agent keeps in its coding um in its memory, the worse its performance is going to be. So you want to make sure that you're not putting too much um context into your agent's memory. And when there is a large amount of context that you're adding you're compacting that in some way. So if you're using a pre-built agent, for example, here in VS Code, it monitors how many tokens or what the context window, how much of that is full in your context window. So for example, there's only seven percent there but you can also use things like uh you know if you're creating your own agent for example here I used Pydantic AI
Speaker 2: To create a compact history function. And so what this function will do is it'll monitor how many tokens there are in the reply so far. And if the history in the message history and if the history reaches a certain amount of tokens, then the agent is going to compact the um the history by maybe just keeping the last two messages And so if you pass through this history processors in the function there in PyDantic AI, it's going to use that to keep track of the context and make sure it's narrow there. Which will make sure the context is um is the the agent continues to respond in a high quality way Now, the second part of
Speaker 2: using our coding agents is going to be about taking action and giving the agent the correct tools that it needs. Here, I would say one of the most popular ways that we're seeing people actually giving agents tools is through MC. Model context protocol, which is an open protocol that was created by Enthropic, and its goal is to standardize the way that we give context to an LLM. And I would say one of the main ways people use MCP is to give your agent access to some tools. So here in when we open our chat, If we click on this button, you'll see that we have a number of different built-in tools with our coding agent. And so these built-in tools are, you know, are great.
Speaker 2: These will usually become, and this is pretty standard in all coding agents, they'll come with things like an edit tool, a read tool, things like a web fetch tool to find information online. But if you want tools that are going to give you access to maybe third-party tools or other things, in this case, for example, I have a bicep tool or you could have a Django tool, maybe you want to connect to Slack, for example, and bring information into your environment for your coding agent to use, you'll select one of these MCP. server. So for example, I can say um I can ask the agent to use the uh excali drill M C P server
Speaker 2: to uh generate a diagram of explaining M C P Explaining MCP. I can misspell and it will still understand what I'm saying, so it's okay. So that's gonna work there. And I will mention as well that Again, in your own code. If you don't want to use one of these and you want to build your own agent, you can also add in directly to your code an MCP server. So here on the side we can see that our agent has started off by you calling the uh Excalibur MCP. server. It's also running this readme, it's using that readme tool. And what we can see is that Excalibur
Speaker 2: provides some information. The MCP server sends in This information on how excalizrole works or how the agent should work with Excalidrol to be able to generate a diagram And then hopefully if it will uh finish on time, the agent would then use that information to make a decision on how to generate the diagram. Sometimes it's a little bit slow this morning. So while it's working, we'll leave it and then come back. So again, that's one way we can use MCP servers to provide tools. Another way we can do this is with something called agent skills. And so with skills, if we go ahead and open that. github folder again, we have this
Speaker 2: skills folder. And skills are a great way. Basically, a skill is just going to be a skill. md file that's saved in a skills folder. And then that uh it's just a mockdown file with a some YAML in the top and that YAML will contain the name of your skill and a description. So skills are a way that you can provide your agent with a dedicated workflow to complete a specific task. And this is really great if you want your agent to, if you're maybe doing something over and over again and you're you want your agent to follow a specific protocol when it is carrying out this skill. So in this example, I have an email skill. I send emails are not my favorite part of my job.
Speaker 2: So I created a skill to be able to Give the agent some instructions on how it should send emails over to my coworkers or something like that at work. For some reason, this is taking a very long time with the escaliodral example here. So we are just going to move forward after the skills section. So the last bit so that we have time to finish off is verification. One thing I want to mention, and I will just Just push this to the side, is that we are seeing a lot of growth in the number in the amount of code that is being pushed out in onto GitHub. And from our numbers, last year in the GitHub State of the Octoverse report, we released the numbers that
Speaker 2: last year was GitHub's most active year ever, and we had about a billion commits pushed to the platform. um in 2025. Sorry, those numbers are slightly wrong. I got an LM to generate it. So this should be 2025 and that should be 2026. here. So you know I I mentioned verification. This is one of the things I should have validated. But last year in 2025 there were a billion commits pushed to GitHub. This year, we're actually on track to seeing 14 times that committed to the platform. And we know that a growing Amount of this code is generated or co-authored by AI agents. So we're actually seeing that in the data at GitHub.
Speaker 2: And this was something that was tweeted out recently by the COO of GitHub, Kyle Daigle. So a question that I think whenever I come to conferences, especially Python conferences, is is AI actually a good thing for developers? Does it actually make people more productive? One of the studies that I found that really helps that I found to be really good on this topic is from AI Engineer last year, and it was a study done at Stanford that Asked the question, can you prove AI ROI in software engineering? And it was a study done on 120,000 developers And what they found was that clean code amplifies AI gains. So if you are keeping your code
Speaker 2: base clean, that's going to make sure, that's going to make it so that you can be more productive. And when AI is actually included in your workflows, it's going to create better code for you. What they also found is that unticked AI in a code base is going to amplify entropy. So they showed in this talk, they showed a specific case study of a company that used AI in their code base but in an unchecked way. And though the number of PRs that this company produced increased. the code quality of that company decreased. And so even though they were using AI and and and the amount of code they were pushing out increased, their output overall wasn't It didn't change that much.
Speaker 2: There was about a 1% increase in their output in terms of actual, you know, what was the quality work that they produced. And a lot of the time that they were using was spent reworking the code. So like we saw on the other slide, maybe fixing tiny mistakes that AI was making. And so a lot of the issues there were spent on refactoring. that code. So I would say specifically what this study shows us is that clean code is actually really important for developers. that are going to be using AI. And some of the priorities for AI-assisted teams should be making sure you have good test coverage. Things like type coverage and quality documentation so the AI knows how to function in your code base. Things like modularity so that changes in one place are not going to affect changes all over your code base
Speaker 2: static code quality and so on. So typically most of the things we see in development today. So the question actually of how can developers create and maintain clean code is not a new one. It's something that we I think I've gone to so many conferences where this was a topic that was debated. And Something that I would say is coming up more and more in circles is test-driven development. And I'm seeing a number of different uh Developers that are using agentic coding, turning to test-driven development as a way to work with coding agents just to make sure that they are validating almost immediately that the code is working as expected So, just a quick overview of what red-green, what it
Speaker 2: means to use a red-green TDD loop. The first thing is that you're going to receive a feature request. Then you're going to go through a red phase where the developer will start by generating tests. And those tests should fail because the feature has not been implemented. After that, you're going to go to the green phase and then you're going to pump out as much quickly as possible, try to make those tests pass. At this stage in the green phase of the loop, you're not really focusing on quality And then at the last stage of this loop, you're going to focus on refactoring. So once the tests have passed, you then go back and make sure that the code that is making those tests pass is high quality enough. And you're also keeping in mind things like uh performance and so on.
Speaker 2: I will say that Historically, with TDD, the go-to has been unit tests and really relying on unit tests to make sure that your code passes. But this can be a big disadvantage when we are working with coding agents because coding agents tend to sometimes write self-affirming tests. So you can give a toting agent a task and it'll say, yes, I correctly did that. And actually it didn't. And it just is is telling itself you're right and you know things are working as expected when they are not working as expected. So how can we actually avoid just focusing on unit tests and creating these sort of self-affirming tests?
Speaker 2: One way that we can do this is with Playwright, and we can use functionality testing for this. So, what exactly is Playwright? Playwright is an open source testing framework that's built by Microsoft. And it automates end-to-end testing in the browser by simulating user interactions. And you can take a look at the Playwright website there. There's different ways to use Playwright with coding agents. So you can use the MCP server. the CLI, or you can use uh playwright agents that will give you three different agent. md files with instructions for how the agent should plan to write tests, generate those tests, and then fix those tests with a healer agent as well. I want to show you quickly, just as we're wrapping up, a T some TDD with Playwright
Speaker 2: with Copilot CLI. And so we're going to switch over here. Oh And we are going to run the command. I want to make sure that this is big enough for us to see. Let's see. Nope, that's probably too big. It's fine. But I'm gonna run the command to get Copilot CLI started. And when we run this command, it's going to start up, it's going to ask me a question, and it's going to load the environment. So the first thing that we want to do is we want to get the requirements or Whatever the feature request is. So sometimes when people are doing test-driven development, they will start with maybe
Speaker 2: if a method has been added to a class, that's what's going to trigger a test but creating a test. But typically here or generally if you're looking at behavioral first or functionality first development, a test, then we want to write a test whenever there's a new feature or quest. So in this example, I got an email. We're pretending that I work for a company called Tailspin Toys. And a few days ago, I got an email from the PM team at Tailspin Toys asking me to add some search and filter features To the website. And they gave me a checklist in my email. And the first thing I did was I sent in a prompt. I said use work IQ to find the email from the PM team with the search and filter spec
Speaker 2: and share the checklist here. So you can see I have I've been asked to add a search bar with Tick search for simple searches and Azure AI search for more complex searches. I've been asked to add a sidebar so customers can filter by category and price. So this is the list of things that I've been asked to build. And one of the cool things is with an MCP server, WorkIQ just allows you to connect to the M365 suite and bring in information there into your terminal. So once I have that, the next thing that I want to do is I want to start by developing a test. And here I sent in a prompt saying, help me develop these features using red-green TDD. Oh Stop by writing playwright
Speaker 2: tests that fail for each feature. Do not commit the changes yet before I can review, just because I'm going to switch over just now. So the first thing the agent is going to do, like we said before, it's going to gather context, it's going to look throughout my code base, and then it's going to actually start to generate those tests. It's going to use the Playwright MCP server that I already have installed to be able to then generate the tests. We can even see it here saying I have a clear picture of the code code base. Let me plan the failing tests and get a quick critique before writing them So it's going to do that, and this is going to take a couple of minutes. But earlier today, I asked the agent to create those same feature and uh the search and filter features in a different um branch and I'm asking it here in a new prompt and in a new terminal
Speaker 2: to actually test those features using Playwright. So we can see that the agent has launched the browser. It's going to go ahead and type in different inputs. So it's looking for Furby and Simon. It's found them correctly. So We can actually see in real time that those features are working in the search bar is working correctly. Now it's also going ahead and it's testing out the different buttons. In the category section, it correctly found the doll, and then now it's checking for the toys in a specific range. So those playwright tests work as expected. So this is a great way if we go from we assume that our agent has generated the correct playwright test for us. Then we actually verified that the code that it's written to
Speaker 2: get these tests to pass is working by actually seeing in the browser. that they work. And then from here I would say the next step would be to go in and actually manually test out the code that the agent has written. To actually refactor the code that the agent has written. So going back and just wrapping up with some few thoughts, these are, it's also just double check if This worked. There we go. We finally got the MCP diagram from Extelidraw to generate in our in VS Code. And again, it just read the context that came from the server and then used the ExcaliDraw tools to draw the diagram. So
Speaker 2: Some last minute uh notes on best practices. I would say some studies have come out recently that have talked about how It's really important for humans to create when you're write using a skills. md file or an agents. md file If a human curates those files, then the performance of the agent is usually going to be better. And in two studies that were published recently that they found that If a human curates a skills file, then the performance is going to be increased by 16%. The performance of the coding agent is going to increase. But if that skill is self-generated by the agent, usually the agent is not going to do a good job.
Speaker 2: And actually that sometimes can have negative effects on the agent's performance because it's going to start overthinking. The same with agents. md file. When a human curates that skill and is careful in creating that agents. md performance increased by four percent there. And then they found that when an LLM generated that for that file, then there was increased amounts of of costs, you know, costs increased by about 20% and the agents started to overthink as well. Some best practices as well is version control. So I would recommend committing often. Once you do something in an agent, if you haven't committed Sometimes it's not gonna remember and then you would have wasted several hours. This has happened to me a lot.
Speaker 2: But if you commit your changes or the changes the agent has made, it's quite good with get so it it will know uh what to do to revert those changes. changes. I'd also say using different branches for different prototypes is also very helpful. Pull requests, something that we're seeing is that maintainers are getting overwhelmed by the amount of code. PRs that are being pushed to GitHub. And so please be mindful with PRs and don't just push to a repo that you have not committed to before. I would also say that it's important to have for critical code a human review that code. And then a final thing I would recommend is when Playwright runs the test. It takes screenshots. Those are great to put in your PRs to help whoever is working with you review those PRs as well.
Speaker 2: So that was that's pretty much it for my talk today. Um yeah, thank you for listening I'm not sure if we have time. I'm not sure if we have time for questions.
Speaker 1: Okay, I'll I'll go afterwards when I get downstairs to you uh
Speaker 3: thank you for the talk. It was really great. Um one question about playwright. Um Playwright tests, because they're high-level, tend to be quite slow compared to low-level tests. And there are so many things and variations you can test when you know clicking around the browser. Do you have a recommendation on basically? How to select what to test? Because I mean they're slow, so you can't have as many as like unit tests.
Speaker 2: Yeah. That's a great question. I think uh Definitely something that with this sort of validation step and verification, a lot of people I think we're struggling to figure out how to speed the process up. And that's um one thing I will suggest is maybe using playwright in headless mode, so not where you actually it launches the browser and you can see it. but where it can run in the background. If you use the Playwright Agents package, it actually has instructions to allow it to run those uh in headless mode and run the tests in parallel. So that should speed some things up. I would also recommend um I think doing things in a modular way might help, but right now I don't know if
Speaker 2: there is a good solution on what to pick versus what not to. I'm not sure if I have a good answer for for that question, but it is slow to an extent versus unit test. But I would say the it definitely when you're using AI generated code, it's going to be more reliable. So I don't really have a great answer to that. Sorry.
Speaker 4: Hi Marlene. Thanks.
Speaker 2: Oh, hey Carlton.
Speaker 4: How you doing?
Speaker 2: Good.
Speaker 4: Um Thanks for the talk. Super. On the quality and the verification side Th these tools are very good at creating a a lot of output very fast. And as a as a developer reading it, it can feel overwhelming to to keep up and and to do that verification. Have you got any kind of Tips for how we either address that or come to terms with it.
Speaker 2: Yeah, so my thoughts on this are let me go back to that's a great question. So with The idea, and this is one of the reasons why I think we need to be pulling from software engineering practices that have already existed and optimizing those, using them for while we're doing AI. So I would say, for example, with TDD, it's supposed to be pretty modular. So I think it's like one test per feature or one test per uh behavior. And that's something I would recommend. So I would say using your AI agent to go quickly through the red and green phase. So having it quickly generate that code for you.
Speaker 2: But also in that step, putting in instructions to be like, try your best to follow. the code the guidance of what already exists or which tests already exist, what the code looks like. You can add in a system prompt for that in copilot instructions. And then I would say spending as much time as you can in the refactor phase. So I think that for developers today, most of your time actually needs to be spent in the refactor phase, not in the red or green, but in refactor. And I would also say focusing on modular changes. So making the changes one at a time so that actually verifying refactoring that code doesn't feel too overwhelming is what I would say.
Speaker 5: Thank you very much, uh Marlene. Um when we're working there are interesting uh interesting and exciting parts of the work and there are boring and dull parts And the error you showed where it had got the dates wrong, that's the kind of boring thing that I'm likely to overlook myself when I'm excited and and working on the big picture How can we turn these all this machinery to to pick up the things that I'm likely to miss because it It's something like last year's date?
Speaker 2: Yeah, that's a great question. Um I think I would I I think obviously I missed that in my in my own slides. So this is something to to be thinking about. But I think the my thought process on this, and and I said in the last slide as well is that I think we should be working with other humans, right? I think that humans it would be amazing to be able to get, especially for critical things. So this small part might not be a big thing But for critical things, it would be great to get another developer to review. Something else, which is a feature that we've released in Copilot CLI, is rubber ducking. So having one agent
Speaker 2: which actually uses one LLM. So maybe you're using the Opus version of models. And then having another agent from another LLM From a different family, so maybe you're using the GPT family of models, follow right after. So one agent creates these changes or makes uh you know creates the code or the slide and then an a a second agent looks at the context around it and then follows behind it and verifies that everything looks okay So ideally that rubber duck system should also work. But I would say if that doesn't work, a human, I really I don't know if how yeah, I think human is only is the only way right now that I can think of Beyond that.
Speaker 2: Great question there.
Speaker 6: Hello Maleen. Thanks for the great talk. I've got a question regarding uh security topics like prompt injections. So uh how resilient are coding agents uh currently against uh prompt injections in open source documentations, for example
Speaker 2: I think with prompt injections, I would so for example if you're working on um Software where you have control over the APIs. Some controls in the Azure APIs, for example, that detect prompt injections and they have like a content safety layer That will detect that and stop the prompt injections from happening. A lot of the labs are building that straight into the models these days where they can detect that something is wrong and it will it will stop um the prompt injection from happening. Beyond that, I would say it's good to have your own guardrails in place.
Speaker 2: So adding some instructions in maybe to um those con those instruction files is a good idea. But I would say if you can, if it's something that's very critical Definitely getting to the API layer, making sure there's contracts control guardrails there in at the model layer is what I would recommend.
Speaker 7: Hi. Um so you showed the research uh about the performance of uh agents MD uh being only marginal Um yeah, that one. So this might be surprising to many people as currently the focus is I think in uh hardness and you know how we can uh optimize their outputs. Do you think we're putting too much um effort into something that has little uh perform um performance gain or will that change in in the near future? Because it uh four percent is uh you know really l little.
Speaker 2: No agree. I mean I think this was for us I think that My personal thoughts on the agent harness topic is that we want to use the harness to be able to help the agent have instructions for when it needs to specialize. So this is comparing, um, it's not necessarily comparing the agent's performance without the the files. It's It's kind of talking about mainly comparing the two states. So I would say that using something, this study was focusing on that. So it wasn't like focusing on what's the ideal agents. md file. It was focusing on
Speaker 2: is it better to have a human-created file versus a an AI generated? I think the harness is good and these sort of files are good for specializing. So I like to think of LLMs are sort of generalists because they're trained on the mean. So they they look at what's most common in in the data. And As a generalist, it's not going to be great at specializing on certain things. And so I think these agents. md, the skills. md is going to be great when you have specific tasks that maybe are specialized to your company, to an industry that you're in. And that's where I think why I think the harness is really important in those cases. Yeah.
Speaker 8: Hello, you have mentioned VS Code only. Uh how does it compare to PyJum in the in the quality of integration?
Speaker 2: In terms of the coding agents with with PyCharm, do you mean uh how does Copilot integrate with PyCharm versus VS Code?
Speaker 8: Yes, uh all the agents.
Speaker 2: I think I've I've try I've tested out the PyCharm version. So Copilot does work in PyCharm. The copilot version in PyTrom, I think, works great. It's slightly, it's sometimes slightly behind VS Code, but it does work great. I've used it before. From what I understand, PyTrom JetBrains has uh something called Juni for uh their own sort of AI chat integration as well. But you can use Copilot in PyTrom and I think it works great. Thank you.
Speaker 7: Regarding MCP versus CLI because some people say MCP is better because uh basically the agent will auto-discover whatever uh but also uh it tends to use a lot of context from the beginning even if you actually want to use it as soon as it's loaded Some people say CLI is better because uh I mean yeah I I there are reasons, you know you cannot answer yes or no. But my question is specific for this use case. Imagine that You have a lot of scripts doing stuff that uh is uh that are being called from your skills and the agent tends to to not follow the instructions and to try you know to to to do uh
Speaker 7: to to to find out the solution by itself without following what has been already uh actually somehow hard coded and it's proven to work but just it it's not being followed because too much context I don't know. Uh
Speaker 8: for this use case, what is your take?
Speaker 7: Would you recommend like MCP or CLI?
Speaker 2: So I've definitely found if you it depends what you're working with. I would say MCP is great if you're working with things that are stateful. For example, here uh with the diagram that we saw was produced, it's now not rendering on the screen, for example. But it with with this, we need an MCP server to be able to return that, you know the the context of a picture to to VS Code and the IDE in the chat. Whereas a CLI wouldn't be able to do that. If you have cases where you are you want to work with the CLI with your coding agent, I actually think sometimes the CLI is more performant.
Speaker 2: Definitely better for search and code or things like that. So I I think it's a bit of a balance. If you can actually do that entire workflow with a CLI, I'd actually recommend doing that And then if your agent isn't following the instructions and the skills. md, you might want to consider some of the compaction strategies, like I mentioned with the Pydantic script. Where we're looking for how many tokens we have so far we're compacting once they reach a certain point. I showed an example where you can keep the last two messages, but you can also have a cheaper You can use an open weight model or something to summarize the context you have so far and then use that as the history. And that's going to probably reduce the amount of context you have in the window.
Speaker 2: Because it should be following the skills and and Even using the CLI at some point is not gonna help if the context window is full. So definitely say work on compaction.
Speaker 7: Hello, uh thank you for the talk. My question is regarding the refactor phase that you mentioned on one of the slides, because I think one of the main approaches currently is to try to do the one-shot thingy.
Speaker 8: And try to do the plan phase for most of your time and then do it in one go, right? So my question is regarding that uh that approach?
Speaker 2: Um so I personally think that the one-shot approach is not good. I'm not a big fan of one-shotting stuff, especially if you want to use Or if you're building something important, um, I'm not a big fan of one-shotting. I do think the planning agent can be good. And I think you can use the planning agent for the red-green phase But um I think when we one-shot, you then have all this code to review at once. And actually with TDD, it's gonna help you break it down so that it's a bit more modular and the code is associated with a test. So what I don't like about one-shotting is that you now have all this code to
Speaker 2: review at once And that seems pretty overwhelming to me unless you one-shot in a modular way. Uh that maybe could could work. But the planning, I still think the planning phase is still relevant at, you know, even with the TDD style approach.
Speaker 8: Hello, I'm here.
Speaker 7: Thank you for the talk. So um
Speaker 8: one thing that always worries me when I'm using agents is that they're always after my in files. So um I tried
Speaker 2: your what? In files. Oh yes, yes, yes, yes.
Speaker 7: So I tried different ways to tell them, okay, don't go to the in files. I tried the it's it's of course in the git ignore. I tried the Exclude I tried so many ways but eventually they go to the N files. So I just removed them completely of the project. So what is the
Speaker 4: most reliable way that I tell uh make the agents to never go to the end files? Thank you.
Speaker 2: uh plug that instruction into your instructions um file and if the agent is not following those instructions it's usually a context overwhelm issue so it's getting too much context So are you using an agent that's like a copilot sort of in the chat? Are you what or are you using like one that you've built yourself? the the one in the chat. Yeah that I would definitely recommend going if you have if it's cursor, cursor rules, if it's copy, copilot instructions, and putting the instructions there. It should already auto compact for you in that case. If the agent does not follow the instructions that are in your copilot instruction file, then it's a bigger issue
Speaker 2: and I'm not sure how unless you actually I'm not really sure how beyond that that you can um because I I don't think it's effective if every single prompt you're s you're giving it that instruction. Usually it should work if it's in the copilot instructions. If not, then We gotta think about some other ideas, which I'm not sure what the answer would be for that.
Speaker 1: Seems like we don't use coding agents at all from the questions. Uh we have two two more questions and I think we'll be done
Speaker 7: Thank you for the nice talk. I have a question about uh environmental effects. Um Given that you're doing test-driven development mi with playwright, you um using a lot of compute power. So how do you think that a single line of code in the output um compares power usage wise if a human has written it or if AI has written it. So can we um justify the increased power usage or does it actually use less power? What's your stance on that Um
Speaker 2: I think that I think AI is using a lot of power. I think there's no getting around that fact. Um and if we're just measuring like human to an LLM, like overall historically, how much energy has gotten the LM to where it is is like so much more than We are using as humans. I think it's difficult because at the end of the day, I'd like to be like I think we have to be open to the fact that We're kind of here and the LLMs are kind of here already. But I do think that a lot of pressure is being put on companies behind these labs
Speaker 2: or you know to do things to offset the the the power usage, the energy usage that they're using with the LLMs. I'm not really sure how much power would be used if we test like an LLM for generating playwright tests in instead of like a human. I don't know what the energy for that one bit of code would be, but I think just in the broader scope of things, my perspective on this is Putting pressure on companies. I know Microsoft has a big sort of employee group that advocates for that and they've done some things to offset.
Speaker 2: the power uses or things like that in the communities that they're in. But it's a it's an ethical issue at the end of the day. And I guess everyone has to kind of decide Yeah, what they want to do.
Speaker 7: Thank you for this talk and for all those answers. I feel scared.
Speaker 8: So uh I I noticed that that you uh started the copilot with the uh dash dash yellow and and and that kind of thing scares me and I mean what's to prevent the agent from changing the agent's MDA or whatever and doing whatever the hell it likes or whatever the hell someone else likes. And um what can you tell me to
Speaker 2: suggest
Speaker 8: to to make me feel more at ease about this
Speaker 2: So I definitely recommend using Git to make you feel more comfortable. This is something I do all the time. I Try to tell the agent, so always when I ask the agent to make changes, I'll say don't commit anything until I review. So I'll make sure the agent is not committing any of those changes. When I'm ready to commit the changes, when I think the agent has done a good job, then I'll I'll I'll let it commit those changes. If not, everything is usually is is pretty reversible. So you can tell the agent to go back to the last And then it should reverse all the changes. And it does that from my experience, it's done that well. Another thing I would say is that My
Speaker 2: personal, even though I think Copilot CLI is fantastic, my personal leaning is to stay in the code editor and use the agent there in the code editor. And I still use YOLO mode in the code editor, but to me it's a little bit better because I can see the files. I can see the diffs, so I can also have the agent list out. And sometimes I do that in the copilot instructions, I'll say whenever you make a change, list out which files you've changed. And so I can go in and I can check the diffs to see uh which changes have been made. If I want things to be reversed, it's easy to reverse them. That's what I would say.
Speaker 1: Thank you very much. Uh I think we're at time. Uh I'm sure we have like tons more of of questions, but uh yeah, I think the coding is here to stay from uh uh what we see. So uh we have like a few minutes now to stretch your legs, stretch your you know uh thank you for being here for being athletes.
April 15 - 17, 2026
Athens, Greece
Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026
Published October 26, 2026