Infomaid
A Textual AI and RAG-Enabled Learning Application
A simple textual AI RAG application using Ollama for generative AI, chatting with PDFs AND chatting (querying) documents prepared in xml!
GitHub link: https://github.com/developmentAC/infomaid
Overview
Infomaid is a simple AI prompt-based solution with built in Retrieval augmented generation (RAG) support!
Welcome to this simple AI application! Infomaid is an experimental AI prompt-driven solution (i.e., each “chat” involves writing a separate prompt to use with a new execution of Infomaid) to help complete text-based work with information.
The software runs locally, without the need to send information for processing to another machine online. This application requires Ollama, for service. Parts of this project code for working with PDFs were borrowed from Pixegami’s RAG tutorial at Reference. Much thanks!
Prerequisites
Before you start, make sure you have the following softwares installed.
Special note: This project was developed on the MacOS (Sonoma). Other OSs such as Windows and Linux have not been tested.
Python
Note: This project was created with Python V3.11. Earlier versions may also be compatible with this project. If you are using an earlier version of Python, then you will have to modify the
pyproject.toml
file from the project.To change from Python v3.10 through v3.13, to just Python v3.9 we make the following change in
pyproject.toml
. Note: This may cause other errors.
Originally:
python = ">=3.10,<3.13"
After editing:
python = "^3.9"
Check to see if you already have Python installed on your machine before installing a new version.
Poetry: Used to manage the virtual environment and to frame the project as a command line app.
Ollama: The AI server.
Set Up Local Models for Ollama
The below commands will install the models that Ollama will require to perform its functions. Note, a typical model for Ollama is about 4 GB in size. As there are two models to install, this project will take about 8 GB of space.
ollama pull mistral
ollama pull nomic-embed-text
Setting Up the Project
We will use Poetry to manage the virtual environment for the project. Use install
to download all the necessary packages for your project.
poetry install
Check that the software is working on your system.
poetry run infomaid --help
Use online help to help you to remember how to use the parameters. Sample commands are privided to copy and paste with editing.
poetry run infomaid --bighelp
Some of the types of parameters are the following.
- Use general chat, give me one result, ask me for a prompt
poetry run infomaid --count 1
- Ask a silly question from the command line! Provide two outputs.
poetry run infomaid --count 2 --prompt "name four shapes"
- Reset and build the internal data of pdf data.
poetry run infomaid --resetdb
Use pdfs as data source with a provided prompt file (
myPrompt.txt
) in which instructions are provided about what the results should look like.- Note: if no prompt is supplied, then the program will stop to ask for one during execution.
poetry run infomaid --usepdfdata --promptfile "promptFiles/myPrompt.txt"
- Query pdfs with prompt. Provide two outputs.
poetry run infomaid --count 2 --usepdfdata --prompt "what is the article's main idea?"
Execution
Parameters
bighelp - Provides CLI commands to use the project.
count - The number of results to give. It is sometimes a good idea to have several results from which to choose as not all output is the same.
promptfile - The parameter to load a text file in which a complicated prompt is provided. The text file can be used to describe the prompt for the outputs using PDF data, in addition to the regular generative output.
model - Used to delare a model if different from the default,
mistral
for general usage (i.e. not for RAG using the PDF data).- Note: the model must first be
pull
ed to the local machine for usage. The command to pull a specific model isollama pull [myModel]
.
- Note: the model must first be
pdfmodel - Use a specific model other than the default
nomic-embed-text
to work with the PDF data.prompt - The initial set of instructions about what kind of information work to complete using Ollama.
usepdfdata - A toggle to specify using RAG support (i.e., PDF data sources). In absence of this parameter at the command line, the Infomaid will not use data from PDF documents.
resetdb - PDF documents are placed in the
data/
directory to provide the content to build a dataset with which the user may interacte. To prepare this dataset after documents have been placd indata/
, the parameter--resetdb
is used to clear out the former PDF content and to update the dataset with the new PDFs.- Note: Always use this parameter when changing PDF content to avoid informational contamination of the results.
Generation
With Infomaid, users may ask the AI to prepare information from prompts such as outlines, emails, and other types of information. Requests can be made with a prompt that may be entered at the command line, inputted after execution, or entered as a text file. The text file may contain large prompts where there are lots of details to consider. In addition, the text file may help to automate jobs where the prompt is created automatically by another task.
Output
All results are placed in the 0_out/
directory. The output files are listed according to the Ollama model that was used to create their content. Automatically created will be a hidden file called .mistral_currentStoryIndex.txt
that will keep track of the file indexing system. If the file is removed, then Infomaid may overwrite the existing files. For new jobs on Infomaid, it is recommended that 0_out/
be removed, or moved somewhere outside of the project directory, in efforts to organize output by prompt.
Working with PDF Data
Infomaid also allows the user to interact with PDF documents to search for ideas which are contained (somewhere) in the documents. Retrieval Augmented Generation, or (RAG), is a natural language processing (NLP) technique that harnesses information retrieval from documents for the delivery of generated information through the use of generative-based artificial intelligence (AI) models.
Sample Project
For instance, imagine that the user wishes to create a draft of a recommendation letter for someone (i.e., a student) who has supplied a current curriculum vitae (CV) as a PDF document. Using Infomaid, a draft of the letter may be written that has been informed by the CV. To use this RAG functionality in this project, the command line parameter, --usepdfdata
, must be utilized to execute the program. See the project’s online help for a sample bash command line scrip to engage the RAG feature.
A prompt for such a task would be the following;
Write a letter of recommendation for MIT graduate school for AstroBill. Use the details from the data to complete the draft.
To set up the project, the PDF of the CV must first be copied into the data/
directory of the project. It is important to note that other non-related PDF documents ought to be removed from this directory to prevent interference with the letter-writing task. The following command is necessary to update the working dataset involving PDFs.
poetry run infomaid --resetdb
The below output will confirm that the database has been updated with the new PDF-derived information
Output:
Resetting database: {resetDB}
Clearing Database
Number of existing documents in DB: 0
Adding new documents: 113
Next, the prompt may be introduced with the following command. Note, this command will return three potential letters that may differ in quality.
poetry run infomaid --count 3 --usepdfdata --prompt "Write a letter of recommendation for MIT graduate school for AstroBill. Use the details from the data to complete the draft."
Same command using the --promptfile FILE.TXT
parameter
poetry run infomaid --count 3 --usepdfdata --promptfile promptFiles/mit.txt
Command Output:
Code prompt: Write a letter of recommendation for MIT graduate school for AstroBill. Use the details from the data to complete the draft. Model: mistral Number of stories to create: 3
The results are Markdown files that will appear in the 0_out/
directory.
Dear Admissions Committee,
I am writing this letter in the highest regard and with great enthusiasm to recommend a remarkable individual, AstroBill, for your esteemed graduate program at MIT. AstroBill, also known as Bill on planet Zirconia, is a multifaceted genius with an insatiable thirst for knowledge and innovation that sets him apart from his peers.
...
In summary, AstroBill's unique blend of intellect, creativity, adaptability, determination, and compassion make him an outstanding candidate for MIT's graduate program. His diverse skills and experiences will enrich any academic or professional environment, and I have no doubt that he will continue to excel in your esteemed institution.
(Or whatever!! Now the letter can be edited to add a human-touch and extra value.)
Testing the Code
The code may be tested to determine functionality. At present, there are two tests; (1) general execution and (2) to determine whether the querying system if working for pdf data.
To run tests using pytest
which is already installed, use the below commands.
poetry install # initialize project
poetry run infomaid --resetdb --usepdf # populate pdf db
poetry run pytest # run tests with the pdf database.
Ethical Note
While there is a lot of convenience in using AI to prepare drafts of letters and other communications, in all this automation, it is important to have a human presence to preside over the generated textual (or graphical work). While AI systems excel at processing vast amounts of data and executing tasks with remarkable efficiency, they lack the nuanced understanding and ethical judgment inherent to human cognition, in addition to the sense of ethics that ought to come from the human world.
Involving ethics in decisions where machines have made the choices (as strange as that may seem) is essential in domains involving communication. Human oversight ensures that communications, whether they involve customer interactions, inter-office correspondence, or public statements, adhere to ethical standards, tone, and context sensitivity. In addition, decisions influenced by AI algorithms must be subjected to human judgment before implementation. Human evaluators can consider broader implications, ethical ramifications, and potential biases that AI systems might overlook. This “human-touch” can therefore help to safeguard against the potential and unintended consequences which may occur at the intersection of data and decision-making, to name one such area.
With this in mind, the Infomaid project must be used responsibly. The project is to serve educational purposes – it is to instruct on the uses of AI, allow for discovery and to entertain (in a way!). Please use Infomaid responsibly.
A Work In Progress
Check back often to see the evolution of the project!! Infomaid is a work-in-progress. Updates will come periodically.
If you would like to contribute to this project, then please do! For instance, if you see some low-hanging fruit or task that you could easily complete, that could add value to the project, then I would love to have your insight.
Otherwise, please create an Issue for bugs or errors. Since I am a teaching faculty member at Allegheny College, I may not have all the time necessary to quickly fix the bugs. I welcome the OpenSource Community to further the development of this project. Much thanks in advance.
If you appreciate this project, please consider clicking the project’s Star button. :-)