"I'm Sorry" is only response from WebSocket client on AWS Ubuntu

[robStewart-128]

Following the steps for the WebSockets example, with some changes, the biggest change being to update to Marian 1.12.0 because version 1.10.0 doesn't compile with the current version of boost (I think).

I spin up a g4dn.xlarge instance (smaller, non-GPU produces same result if you want to save some money), with a 40G HDD, then login via SSH and issue the following commands...

cd ~
sudo apt update
sudo apt upgrade -y

# You may need to say "yes" for a particular update, and potentially reboot.
# If you did reboot...
cd ~

sudo apt autoremove
sudo apt autoclean
sudo apt install git make gnupg2
git clone https://github.com/Helsinki-NLP/Opus-MT.git
cd Opus-MT/install

# Edit the "Makefile" in the install folder to checkout 'marian 1.12.0'.
#
#marian:
# git clone https://github.com/marian-nmt/marian
# cd marian && git checkout 1.10.0
# ...TO (without the hashes)...
#marian:
# git clone https://github.com/marian-nmt/marian
# cd marian && git checkout 1.12.0

make all
# (Select 'y' for yes when asked if you want to install stuff).

sudo make install
cd ..
sudo make all

service marian-opus-fi-en status
service opusMT-opus-fi-en status
service opusMT status
# All seems good.

echo "Mitä kuuluu? Käännös on hauskaa." | ./opusMT-client.py -H localhost -P 20000 -s fi -t en

# Response is not as expected...
▁ # ▁I ' m ▁sorry , ▁I ' m ▁sorry , ▁I ' m ▁sorry ||| 0-0 1-1 4-2 4-3 4-4 4-5 7-8 7-13 8-7 8-12 10-9 10-14 11-10 11-15 12-6 12-11 12-16

Any suggestions would be appreciated. Maybe it's obvious, but I'm just trying to follow the instructions?
Rob

[robStewart-128]

For those who need AWS Ubuntu, a GPU and Opus-MT, here's how I finally got things to work...

Spin up a g4dn.xlarge instance with, say 50GB of HDD. Connect to the instance, and do the following...

sudo apt install -y make gnupg2 cmake build-essential libboost-system-dev libprotobuf23 protobuf-compiler libprotobuf-dev openssl libssl-dev libgoogle-perftools-dev intel-mkl-full

Reboot and reconnect, if required or recommended by the above.

mkdir ~/cuda
cd ~/cuda
wget https://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_535.54.03_linux.run
sudo sh cuda_12.2.0_535.54.03_linux.run

‘accept’ the EULA

Confirm “Driver”, and version “535.54.03” are selected.

Confirm “CUDA Toolkit 12.2” is selected.

Deselect “Demo Suite”, and “Documentation”.

Confirm “Kernel Object” and “nvidia-fs” are not selected.

Select “Install” and hit enter.

cd ~
rm -rf ~/cuda

git clone https://github.com/marian-nmt/marian
mkdir -p ~/marian/build
cd ~/marian/build
cmake .. -DCOMPILE_SERVER=on -DCOMPILE_CPU=off -DCOMPILE_CUDA=on -DUSE_STATIC_LIBS=on -DUSE_SENTENCEPIECE=off
make

mkdir -p ~/.local/bin/
mv marian* ~/.local/bin/
cd ~
rm -rf ~/marian

git clone https://github.com/Helsinki-NLP/Opus-MT.git
sudo apt install -y python3-pip
cd ~/Opus-MT
pip install -r requirements.txt

Use the default models, or install appropriate models - I leave that to the reader. (You will need to update ~/Opus-MT/services.json if you do change models).

echo $PATH

If, and only if, the “.local/bin” is not present in the result from the above…

export PATH=~/.local/bin:$PATH

python3 server.py

In your AWS EC2 console, adjust the inbound rules for the instance security group to allow port 8888 from your computer's IP.

From your computer issue a request like the following - but first replace the "[domain name]" token with the actual domain name of your instance.

For clarity - "your computer" and "your instance" are NOT the same thing. Your computer is your work or home PC, your instance is the EC2 you spun up at the start of this doc.

curl --noproxy "*" --request POST --header "Content-Type: application/json" --data '{"from":"fi","to":"en","source":"Mitä kuuluu? Käännös on hauskaa."}' http://[domain name]:8888/api/translate

That should work! Or at least get you most of the way there. NOTE - Some packages may NOT be required, but I haven't tested all options.

And there may be better ways to install CUDA drivers etc. but that I also leave to the reader to discover.

Rob