Random binary data is encoding differently

[keith-turner]

While working on #3 I was running some test. I tried running the following commands (using commit b51529a) and there were lots of differences.

~/git/ecoji-js$ head -c 100 /dev/urandom > test.dat
~/git/ecoji-js$ cat test.dat | ecoji -w 0  | xxd > ecoji-go-xxd.txt
~/git/ecoji-js$ cat test.dat | node ./bin/cli.js  | xxd > ecoji-js-xxd.txt
~/git/ecoji-js$ diff ecoji-go-xxd.txt ecoji-js-xxd.txt 

If I convert the random binary data to base64, then there are no diffs except for a new line diff.

~/git/ecoji-js$ cat test.dat | base64 -w 0 | node ./bin/cli.js  | xxd > ecoji-js-xxd.txt
~/git/ecoji-js$ cat test.dat | base64 -w 0 | ecoji -w 0  | xxd > ecoji-go-xxd.txt
~/git/ecoji-js$ diff ecoji-js-xxd.txt ecoji-go-xxd.txt 
28c28
< 000001b0: f09f 8f81 e298 95e2 9895 e298 950a       ..............
---
> 000001b0: f09f 8f81 e298 95e2 9895 e298 95         .............

So I suspect there is something going on with binary data. This may be because string is used in the source instead of byte array. I don't know javascript well enough to say. If this is a limitation of javascript, a disclaimer in the readme might help.

[dimabory]

@keith-turner Thanks for the feedback.

It's really odd to me why it has so lot of differences.

This may be because string is used in the source instead of byte array.

I'm using Buffer which implements Uint8Array, so I don't think the main reason is in it.

I would be glad to receive your any help to figure out an issue eventually.

[dimabory]

@keith-turner Please take a look at examples provided how binary data being parsed in Node and Go. Not sure why Node is not compatible with Go's rune behaviour...

Node https://repl.it/@Brsvsk/node-buffer-test
Go https://repl.it/@Brsvsk/go-buffer-test