Quantized metal.

[Narsil]

Working quantized state for candle.

High level overview:

• Introduce QStorage to split Metal and Cpu ops.
• Quantize/Dequantize still running on CPU even when asked to run on metal. Kernels exist but led to different quantization/dequantization. GGML removed those kernels on device too (I guess because of the differences).
  I think we should keep the surface for on metal quantize/dequantize so we can easily implement them later. They are part of the ggml API imho.
• Added a bunch of test, using test_device! in order to get similar testing behavior as regular Tensor.
• quantized.metal is a direct copy of ggml's ggml-metal.metal. This choice was made so further dev could be made faster and bugs mentionned after can be imported more easily. All the glue logic is in candle_metal_kernels.
• Introduces a new GgmlDType in candle_metal_kernels. Ggml uses different kernels based on size of matmul and hardware capacity. This wasn't implemented here, but could with the current API.

Worthy bug already discovered (not fixed in this PR since they do not belong here):

• Q2K Metal -> Bugged (also present in GGML).
• Q4K CPU -> Bugged (present reviously, new test catch it).
• Q5K CPU -> Bugged (present previously).
• Q8_1 Both -> Never really implemented it seems
• Q8K metal -> Never implemented in metal

[LaurentMazare]

All the ggmldtype bits seems like an orthogonal refactoring that probably is orthogonal to metal? Could this be split in a separate PR? Also all the fences bits seem orthogonal too and could be extracted.

[LaurentMazare]

Also blck_size is not a typo, it's on purpose that it matches the llama.cpp nomenclature.

[Narsil]

All the ggmldtype bits seems like an orthogonal refactoring that probably is orthogonal to metal? Could this be split in a separate PR? Also all the fences bits seem orthogonal too and could be extracted.

No it's not. It's core to it. The reason is that a lot of the previous code was using the GgmlType (the block type, not the dtype) as a generic. This doesn't work for the metal bit since the buffer that actually store the data are untyped (unlike Vec), therefore we need to change that around. (I know we could use PhantomData, but seems very anti-pattern here, and overall the code seems much simpler like this).

For blck_size, I'm ok respecting llama.cpp convention, but to a point, here it's almost silly not writing it all out.

[Narsil]

Merged #1523 directly.