Develop demo API

[secretsauceai]

Description

Similar to the Fast API for the LLM chatbot that was developed, we should develop a demo API and include that in this repo. This will help people try out the model that will be available.

NOTE: This issue is quite large, once it is refined, it will probably be broken down into several issues.

DoD

• Adapt the inference script for API usage
• Look into what format the input-output should have
• Develop an API that takes the user input, formats it correctly with the prompts, runs inference, and sends back the results
• Test API
• Get feedback
• Make any improvements
• Document