Show HN: Rust-based translator with KULYK models for EnglishUkrainian MT

3 weeks ago 3

Hostinger Web Hosting

This is a Rust-based translator that utilizes the llama-cpp-2 crate to inference Kulyk models.

Created for the Rustcamp 2025 Summer: https://github.com/rust-lang-ua/rustcamp_ml

Demo

Models

Build

cargo build --release

Cross build

Build images

docker build --platform=linux/amd64 -f dockerfiles/Dockerfile.aarch64-unknown-linux-gnu -t aarch64-unknown-linux-gnu:my-edge . docker build --platform=linux/amd64 -f dockerfiles/Dockerfile.x86_64-unknown-linux-gnu -t x86_64-unknown-linux-gnu:my-edge .

Build binaries

cargo install cross --git https://github.com/cross-rs/cross goreleaser build --clean --snapshot --id kulyk --timeout 60m

Usage

# this command will download quantized GGUF models just download_models # start web server and navigate to http://localhost:3000 in your browser just run

Run using Docker

docker run -p 3000:3000 --rm ghcr.io/egorsmkv/kulyk-rust:latest

High-Load test

Test set: https://huggingface.co/datasets/speech-uk/text-to-speech-sentences
Threads: 16
GPU: NVIDIA GeForce RTX 3090
Memory usage: 1190MiB
Average inference speed per sentence: 0.0537 sec = 53.7 ms

Acknowledgements

This project is based on the following repositories:

simple example from llama-cpp-2

Read Entire Article

Hostinger Web Hosting