Skip to content

Commit e1467c8

Browse files
committed
**Update VS Code extension development guide with AI usage and limitations**
- Revise Git branch naming conventions and AI-generated file management - Add AI as a judge for diff validation and test visx in codespace - Update context window visibility, rate limits, and system prompt design - Adjust CI considerations and conclusion for GitHub Copilot's agent mode
1 parent cd610d7 commit e1467c8

3 files changed

Lines changed: 13 additions & 19 deletions

File tree

content/en/blog/developing-a-vs-code-extension-with-github-copilot-agent-mode-and-claude-4-sonnet-may-2025.md

Lines changed: 10 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ To enable Claude 4 Sonnet:
6666

6767
### 1. Git Branch Naming
6868

69-
By default, the cloned repository uses `master` as the main branch. To align with modern conventions:
69+
By default (claude sonnet-4 prompt without details about the branch name), the cloned repository uses `master` as the main branch. To align with modern conventions:
7070

7171
```bash
7272
git branch -m master main
@@ -81,47 +81,42 @@ Claude 4 Sonnet tends to generate `.md` files for various outputs, which can clu
8181
* Implement a cleanup script that removes outdated `.md` files not matching the current codebase.
8282
* Run this script as part of your build or deployment process to maintain a clean workspace.
8383

84-
### 3. Managing Large Diffs
85-
86-
When applying diffs suggested by AI models:
87-
88-
* **Reliable Diff Application**: Writing changes to files and applying diffs works reliably with minimal issues in this setup. Performance is generally stable and fast, especially when files are properly split into smaller modules rather than large monolithic ones.
89-
* **Split Large Files**: Break down large files into smaller modules to prevent timeouts or memory issues.
90-
* **Incremental Application**: Apply diffs incrementally across modules to ensure stability and reduce errors.
91-
92-
## AI as a Judge (Diff Validation)
84+
## AI as a Judge / Additional AI usage
9385

9486
To assess the quality of AI-generated changes:
9587

9688
* **Use Roo Code**: Utilize [Roo Code](https://docs.roocode.com/features/custom-modes) in its **chat interface**, paired with a custom mode configuration and Gemini 2.5 Pro/Flash as the backing model. The `.roomodes` used in this project are tailored for tasks such as additional validation, Git operations, and technical consulting. You can review the configuration here: [roomodes file](https://github.com/d-oit/do-vscode-lm-api-explorer/blob/main/.roomodes).
9789

9890
This enables high-level, context-aware evaluation of your implementation without needing to paste diffs or use any CLI tools. For more, see [Roo Code Chat usage](https://docs.roocode.com/basic-usage/the-chat-interface).
9991

92+
And important: Test the **visx** in codespace or on other system.
93+
10094
## Limitations and Known Issues
10195

102-
### 1. Context Window Visibility
96+
### 1. Context Window Visibility / Agent mode
10397

104-
GitHub Copilot does not currently expose information about the context window size or token usage during interactions. This lack of transparency makes it difficult to manage prompt size, debug unexpected truncations, or optimize for longer code generation tasks.
98+
GitHub Copilot does not currently expose information about the context window size or token usage during interactions. This lack of transparency makes it difficult to manage prompt size, debug unexpected truncations, or optimize for longer code generation tasks. You have to press "Continue" for the next step in Agent Mode - not a complete agent mode without interactions.
10599

106100
### 2. Rate Limits and High Load Conditions
107101

108102
Claude 4 Sonnet on GitHub Copilot may become temporarily unavailable under high load, particularly during peak usage times. While most operations work smoothly, occasional delays or fallback to other models may occur. Best practice for me: wait, do not change to another model. Try again with retry or chat with "continue".
109103

110104
### 3. System Prompt Design (Hint)
111105

112-
Although GitHub does not publicly document the system prompt structure, those interested in how GitHub Copilot might structure its system prompts can refer to this Postman trace capture: [example gist](https://gist.github.com/d-oit/231f6f949cad9ea8d1804ee047581ca6).
106+
Although GitHub does not publicly document the system prompt structure, those interested in how GitHub Copilot might structure its system prompts can refer to this Postman trace capture: [example gist](https://gist.github.com/d-oit/231f6f949cad9ea8d1804ee047581ca6). You could not customize the system prompt or set another temperature like e.g. in Roo Code
113107

114108
## Continuous Integration (CI) Considerations
115109

116110
The existing CI pipeline may encounter issues due to:
117111

118112
* **Branch Naming**: Ensure your CI configuration references the correct branch (`main` instead of `master`).
119-
* **Copilot Chat Dependency**: End-to-end (E2E) tests that depend on GitHub Copilot Chat integration are challenging to automate and remain on the TODO list for future implementation.
113+
* **Copilot Chat Dependency**: End-to-end (E2E) tests that depend on GitHub Copilot Chat integration are challenging to automate and remain on my TODO list for implementation.
120114

121115
## Conclusion
122116

123-
I'm genuinely impressed by the current state of GitHub Copilot's agent mode, especially when paired with Claude Sonnet 4. The improvements in functionalities like "apply diff" and codebase search are substantial compared to earlier versions this year. However, it's important to note that this enhanced experience is available without limitations only until June 4, 2025.
117+
I'm genuinely impressed by the current state of GitHub Copilot's agent mode, especially when paired with Claude Sonnet 4. The improvements in functionalities like "apply diff" and codebase search are substantial compared to earlier versions this year. However, it's important to note that this enhanced experience is available without [limitations only until June 4, 2025](https://docs.github.com/en/copilot/managing-copilot/monitoring-usage-and-entitlements/about-premium-requests#premium-requests).
124118

125119
Previously, using Claude Sonnet 3.7 within GitHub Copilot presented challenges, including frequent rate limiting and inconsistent performance. These issues often hindered productivity and made the development process less efficient.
126120

127121
With the introduction of Claude Sonnet 4, many of these problems have been addressed. The model offers a more reliable and efficient coding assistant experience, making it a valuable tool for developers.
122+
I was particularly impressed by the speed and error-free writing of file changes compared to before.

content/en/projects/do-vscode-lm-api-explorer.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ draft: false
2525
Source Code
2626
{{< /button >}}
2727

28-
More infos about the project: [Blog article](/blog/get-licenses-from-used-nuget-packages-for-your-.net-core-solution)
28+
More infos about the project: [Blog article](/blog/developing-a-vs-code-extension-with-github-copilot-agent-mode-and-claude-4-sonnet-may-2025)
2929

3030
## Github readme
3131

hugo_stats.json

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -523,7 +523,7 @@
523523
"030---improved-packaging--vs-code-version-support",
524524
"031---enhanced-disposable-management--test-environment-improvements",
525525
"1-capturing-with-postmans-built-in-proxy",
526-
"1-context-window-visibility",
526+
"1-context-window-visibility--agent-mode",
527527
"1-git-branch-naming",
528528
"1-when-copilots-generate-commit-hits-its-limit-or-windsurf-is-only-available-for-users-on-a-paid-plan",
529529
"2-grab-your-free-codestral-key",
@@ -534,12 +534,11 @@
534534
"2024-11-22-add-previous-next-button",
535535
"3-capturing-with-mitmproxy",
536536
"3-dropin-shell-script-ai-git-commitsh",
537-
"3-managing-large-diffs",
538537
"3-system-prompt-design-hint",
539538
"4-feeding-vs-code-through-your-proxy",
540539
"5-digging-into-your-logs",
541540
"Copyright",
542-
"ai-as-a-judge-diff-validation",
541+
"ai-as-a-judge--additional-ai-usage",
543542
"automating-playwright-tests-from-a-sitemap-with-mistral-ai",
544543
"backToTop",
545544
"backend",

0 commit comments

Comments
 (0)