Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GTE-style fixed-point math #14

Open
zole opened this issue May 24, 2022 · 0 comments
Open

GTE-style fixed-point math #14

zole opened this issue May 24, 2022 · 0 comments
Assignees
Labels
accuracy enhancement New feature or request

Comments

@zole
Copy link
Contributor

zole commented May 24, 2022

Currently we round the results of matrix multiplications, but the math is still done in floating point on the GPU. Per the spec the GTE did fixed point math. Would this be too much accuracy (on top of probably being slower)?

@zole zole added enhancement New feature or request accuracy labels May 24, 2022
@zole zole self-assigned this May 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accuracy enhancement New feature or request
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant