Discussion about this post

User's avatar
Neural Foundry's avatar

Fantastic work on achieving competetive performance at 8B parameters while enabling local deployment. The privacy angle is what really matters here tho - companies sitting on sensitive threatdata can finally run decent security analysis without shipping everythign to external APIs. I tried deploying similar solutions last year and the GPU requirements were wild, interested to see if this actualy runs smooth on consumer hardware.

No posts

Ready for more?