Skip to content

Fix checkpoint policy for Grok#76

Open
hx89 wants to merge 4 commits intogoogle:mainfrom
hx89:fix_checkpoint
Open

Fix checkpoint policy for Grok#76
hx89 wants to merge 4 commits intogoogle:mainfrom
hx89:fix_checkpoint

Conversation

@hx89
Copy link
Contributor

@hx89 hx89 commented Jun 18, 2024

Fix two issues:

  1. Checkpoint policy is not passed into Grok PP
  2. Bug in praxis attention layer, causing qkv proj output not checkpointed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant