feat: suport hybrid pool kvcache allocator#943
Open
SJTUGavinLiu wants to merge 8 commits intomainfrom
Open
Conversation
ce065cc to
e454000
Compare
Collaborator
AI Code Review - PR #943Status: LGTM Summary: P0/0 · P1/0 · P2/3 · P3/3 lgtm ready to ci Non-blocking SuggestionsP2
P3
Review Checklist: 6 pass / 6 failGeneral Principles ChecklistFailed
Passed
RTP-LLM ChecklistFailed
Passed
Python Static-First ChecklistPassed
Strengths
|
Collaborator
AI Code Review - PR #943Status: LGTM Summary: P0/0 · P1/0 · P2/3 · P3/2 lgtm ready to ci Non-blocking SuggestionsP2
P3
Review Checklist: 8 pass / 3 failGeneral Principles ChecklistFailed
Passed
RTP-LLM ChecklistFailed
Passed
Python Static-First ChecklistPassed
Strengths
|
Collaborator
AI Code Review - PR #943Status: LGTM Summary: P0/0 · P1/0 · P2/2 · P3/3 lgtm ready to ci Non-blocking SuggestionsP2
P3
Review Checklist: 5 pass / 4 failGeneral Principles ChecklistFailed
Passed
RTP-LLM ChecklistFailed
Passed
Python Static-First ChecklistPassed
Strengths
|
Add per-layer region name awareness to PD separation cache transfer. Refactors DecodeRpcServer and ExecOps to use layer_region_to_group mapping, and extends makeCacheKey to include region_name suffix. Linear groups now transfer the last two blocks instead of one.
Extend KVCacheMemoryConnector to use layer+region_name slots for block layout, copy plan building, and validation. This allows models with multiple cache regions per layer to correctly transfer blocks through the memory connector path.
e799c16 to
23fae48
Compare
Collaborator
AI Code Review - PR #943Status: LGTM Summary: P0/0 · P1/0 · P2/1 · P3/3 lgtm ready to ci Non-blocking SuggestionsP2
P3
Review Checklist: 7 pass / 4 failGeneral Principles ChecklistPassed
RTP-LLM ChecklistFailed
Passed
Python Static-First ChecklistPassed
Strengths
|
Collaborator
AI Code Review - PR #943Status: LGTM Summary: P0/0 · P1/0 · P2/0 · P3/3 lgtm ready to ci Non-blocking SuggestionsP3
Review Checklist: 7 pass / 2 failGeneral Principles ChecklistFailed
Passed
RTP-LLM ChecklistFailed
Passed
Python Static-First ChecklistPassed
Strengths
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
for cr