Skip to content

[draft] Fp8 inference experimental#72

Open
wenscarl wants to merge 3 commits intogoogle:mainfrom
wenscarl:fp8_inferene_experimental
Open

[draft] Fp8 inference experimental#72
wenscarl wants to merge 3 commits intogoogle:mainfrom
wenscarl:fp8_inferene_experimental

Conversation

@wenscarl
Copy link
Contributor

@wenscarl wenscarl commented Jun 5, 2024

Fp8 matmul adopts per-tensor, symmetric and activation+weight quantization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant