Here we will discuss the steps to install LLM in local machine on Windows:
Steps:
- Goto https://ollama.com/download/windows
- Download & install the Ollama platform on your machine by clicking the OllamaSetup.exe file
- Once the installation is completed, to check the Ollama platform is running navigate to http://localhost:11434/ & check the status.
- If the platform is installed successfully it will show :Ollama is running
- Now we need to install the language model.
- Navigate to https://ollama.com/library
- You can now install any language model based on your need
- To run the language model, goto powershell prompt
- Run the command ollama run <language_model_name> e.g. ollama run llama2
- Once the LLM is installed, it will provide a command prompt to interact like Chatgpt
- The API of Ollama can be used to interact using Spring AI
Further readings:
https://thenewstack.io/how-to-set-up-and-run-a-local-llm-with-ollama-and-llama-2/
https://www.kdnuggets.com/ollama-tutorial-running-llms-locally-made-super-simple
https://www.youtube.com/watch?v=5ecArhs6d7I&pp=ygULamF2YSB0ZWNoaWU%3D
Sample Codebase:
https://github.com/Java-Techie-jt/spring-ai-llama2/tree/main
