Ollama -

Ollama is how you can run a LLM in your home, without the internet. Trust me, it is fun.

  1.  Installing Ollama

    1. Hardware!
      • Pick the model that you want to run. If the model is large, then you will need a large amount of RAM. The size of the model is measured in Parameters. One billion parameters will require one Gigabyte of RAM. 16B parameters requires 16GB of RAM. 500M parameters requires 500MB of RAM. 
      • Then you will need a NVIDIA GPU which has a "Compute Complexity" greater than 5. 
        • Why? The embedding space is where each word and syllable is mapped using a large amount of dimensions. Imagine each word as a point floating in space. From the center of space, each point is like a radius with its various angles. An easy way to compute how close each point is to all other neighbouring points is by doing the same mathematics (Cosine Affinity) that the NVIDIA GPUs instruction sets can do across hundreds of cores, while your more complex CPU with its handful of cores can do these maths instructions and much, much more, due to the CPU managing your computer.   
    2. Software!
      • Well, www.Ollama.com asks you to install a  peice  of software. The Ollama packaige which is a Client+Server bundle. The Client is how the user talks to the LLM. The Server manages the LLM, loads the model, does the inference, returns the answer. 
      • Ollama also uses "Go" a run-time language created by Google.com. At www.go.dev So, like Java, you may want to download to Go package, which installs its runtime and compiler. 
      • To test if you installed Ollama, open your console and try "ollama --help"
      • To test if you installed  Go, open your console and try "go -version" 
  1. Downloading a model
    1. Find the list of models at www.ollama.com/library 
    2. Pick a model based on your needs. There are models for the linguistic arts, models for image recognition, and models for coding and maths. 
    3. in your console, try "ollama run modelname' 

  1. Running your model

    • The Server is an HTTP server that waits on your computer when your computer starts up... like a trojan. 
    • When you call "ollama ps" in the cammandline. (Powershell), ollama is a HTTP client like a browser that contacts the server. 
    • You can always type in "ollama run 'model name' because if you have downloaded it once it will be saved with you. This starts a 'chat' client, in your commandline interface.
    • Once you see ">>>" you can type "/help"(and enter) to explore what options you have. 
  1. Your client -
    1. Because you are using HTTP , and HTTP is very human-friendly, you can create your own client to chat with an Ollama model.   https://github.com/ollama/ollama/blob/main/api/client.go
    2. The word "Chat" here is different from "Generate". "Chat" bundles up your past request-response pairs as history of the Chat. Generate doesn't bother to fill your 'messages' with history. You can read the types.h file to see the different structures, for which Chat Response and Chat Request are two different structures.    https://github.com/ollama/ollama/blob/main/api/types.go
    3. If you want to use "tools" in Ollama, then you need to build your own client interface, because the model will respond with a request for a tool, and you are free to let your client respond as the assistant with the results of a function tool, back to the model, as an intermediate step between your Request, and the Model's Response. 
  2. Prompts
    1. Properly formatting your prompt can help.  There are 7 categories in this Cognitive prompt structure, which is probably more help than hint. https://arxiv.org/html/2410.02953v2
  3. Tools
    1. Models with tool support have a special token, which is an "unknown" or "i dont' know" token. 
  4. Creating a new model
    1. TEMPLATES
      1. Templates use Go Lang's template thing which is a little like PERL or PHP as a text manipulation tool. And The templates can manipulate the .Prompt which is the User's input, and can do manipulation of  the .Response which is the LLM's output. However, the key is that the Template can use Tools. And Tools can be an external program like Python or C++ that the Template calls for either.  
    2. Modelfile
      1. The model File allows you to create your own model, using an encoded model as the base. The model file allows you to tweak the parameters. The embedding space of a word or token can be imagined like a flower with the center  of the flower where the stem is being the word in question, and all other words being the tips of the petals. The parameters tell the machine how open the flower can be, from fully outstretched and wonky to tightly curled up and repetitive. 
    3. The source. 
      1.  The source according to ollama.com is at github.com. Within the source, there is:
        1.  the client
        2. The server
        3. The llama.cpp machine 

Comments

Post a Comment

Popular posts from this blog

The smallest ℝeal number in Mathematics and Αlpha and Ωmega

Image
The smallest number in mathematics is invisible to science, but geometrically looks like Jesus Christ's two Greek-letters "Alpha" and "Omega". contents Defining 1/infinity Appreciating Α Ω Takeaway summary What Jesus said The effect of What Jesus said Calculus uses a special number δx = 1 ÷ infinity which is the smallest number imaginable. It is 0.0000…00001. if 1÷10 = 0.1 and 1/100 = 0.01 then 1/infinity = 0.0000000000…001 . “dx” is shorthand for the Greek letter Delta x = Δx or δx , because delta "Δ" is used in mathematics to mean ‘change in’ . δX is spoken as "d X" Revelation John Greets the Seven Churches 4John to the seven churches which are in Asia: Grace be unto you, and peace, from him which is, and which was, and which is to come; and from the seven Spirits which are before his throne; 5And from Jesus Christ, who is the faithful witness, and the first begotten of the dead, and the prince of the kings of the earth. Unto ...
Reveal Headlines »

Jesus' "love your neighbour as yourself" law and Calculus integration.

Image
The relationship between Lord Jesus' double law "Love eternal God and Love thy neighbour" and Calculus' integration.  Take-home lessons:  To get to  the higher dimensions of life , we must love one another.  To love one another you each must love your neighbour as yourself.  You must treat your neighbour as you want others to treat you.  You must change your view of your self, to accomodate the others as they accomodate you, and not simply just view yourself as by your self and next to other people.  By accomodate i mean, to tolerate, to accept, to get along with, to work with, to help, to be helped by, to share the world with.  The infinitesimal unit is an individual.  In geometry and in calculus, this is easily visualized. A number  is a summation of many "ones." In geometry a number is a line.  A line is a summation of lots of (1/infinity big) dots in sequence.  Each dot has two n...
Reveal Headlines »

Prompt-driven Programming

Fine-tuning prompts for programming tasks involves clearly defining the problem, specifying the input and output format, and sometimes providing examples. Here's how you can do it: examples edge case sequences ambiguities clarify the problem specify inputs specify outputs comments 1. **Define the Problem Clearly:** Be clear and concise about what you want the program to achieve. The problem statement should be explicit about what the task involves. 2. **Specify the Inputs:** Clearly define what the inputs are and in what format they will be given. This may include the type of the data, the range of possible values, and any other relevant information. 3. **Specify the Outputs:** Similarly, be explicit about what the outputs should be. Describe the format that the output should be in, whether it's a certain data type, a particular data structure, a specific range of values, etc. 4. **Provide Examples:** Provide examples of input and output pairs. This can help to c...
Reveal Headlines »