In the constantly evolving universe of artificial intelligence and natural language processing, txtai emerges as a revolutionary tool, especially for those delving into the world of large language models (LLMs). Imagine a bridge between you and an array of powerful LLMs, where txtai plays the role of a facilitator, simplifying and amplifying your interactions with these complex technologies.
Extended Connectivity with LLMs
Txtai stands out with its ability to connect to various LLMs. Whether it's GPT-3, BERT, or other cutting-edge models, txtai acts as a unified channel, allowing you to tap into these computational powers without the hassle of complex configurations. For developers, this means a significant reduction in the time and effort required to integrate these models into their applications.
Preconfigured Features for Ease of Use
One of the major strengths of txtai is the pre-configuration of its diverse text processing functions. Classification, summarization, translation, and many other NLP tasks are ready to use, requiring only minimal developer intervention. This ease of use makes txtai particularly appealing for those wanting to exploit the power of LLMs without diving into technical details.
Customizable Workflow for Batch Tasks
The real magic of txtai lies in its ability to batch tasks into customizable workflows. This opens immense potential for businesses by enabling them to chain NLP operations into logical sequences, thereby automating complex processes. Whether it's for sentiment analysis, generating document summaries, or conducting in-depth searches, txtai makes these tasks not only possible but also surprisingly simple.
Intelligent Searches and More
Beyond basic textual searches, txtai excels at conducting intelligent searches that understand the context and nuance of queries. This ability to intelligently interpret and respond to search requests makes txtai not only a powerful tool for developers but also a strategic asset for businesses looking to maximize textual data.
In summary, txtai is not just a tool; it's a facilitator, an amplifier of possibilities in the field of artificial intelligence and natural language processing. With txtai, developers have unprecedented access to a range of advanced NLP features, all while enjoying exceptional simplicity and adaptability.
For a very quick example of what txtAi can do in a few lines, here's a first example from the official documentation: https://github.com/neuml/txtai/blob/master/examples/01_Introducing_txtai.ipynb
# Example Python code here
We start a server:
# Command to start the server
And voila, directly wired with FastApi, you have an API ready to use
http://127.0.0.1:8000/docs gives you direct access to a swagger space to play with your configuration.
Now let's launch a request on our API and analyze what happens:
# Example request code
The response will be:
# Example response
But then, what happened? Remember the example workflow:
# Workflow example
Through the pipeline, we ask the Model to "translate into French if the message is English", we pass the output of this task directly to the next task: "Tell me the language of this text". In our payload, the first sentence is in English, so the model will translate it for us; in the second, it's in German, it won't translate it. But then why doesn't it recognize the German language? In fact, the official example uses google/flan-t5-large while I used the flan-t5-small. Less accurate. I try again with the t5-large and it will give me
[
"French",
"German"
]
That's better, with a slightly larger model, the accuracy is better. The larger your model, the more you're going to need a GPU with RAM. There's a good ratio to find between the necessary hardware and the desired response quality.
So, you will have understood, with very few lines of code, you have an API that can do a whole bunch of interesting things for you. You can index your PDF files, your images, all the documents of your company and make intelligent queries on them. Or even build a database for questions/answers and query a semantic search engine in the language of your choice for the language of your choice. The possibilities are only limited by your imagination/need.
So all of this is great, but if we need a little more flexibility. We can see that in the documentation the list of Tasks is large and most things are done automatically. Except that we know well. We are rarely in a case where everything is already there and that's where you have to start doing things yourself by hand.
So for the example, we will create our own workflow, with our own task on an endpoint we will have even imagined. So here's what we want:
- An API of type: /translate in POST with a payload of this type:
# Example JSON payload
JSON.stringify($('#test_json').html()) on any site will give you content for the example.
- We thus want to retrieve HTML content and translate it. And finally, of course, return the result.
You are going to tell me, it is super simple. And you will not be entirely wrong. Because it is, indeed. But as with everything, once we have an example in front of us, it's very simple. Since simple examples are scarce, I still had to search and fiddle a bit.
I therefore propose that you read the code, and the comments should guide you on how to proceed. So we have 2 files: main.py and TranslateHtml.py. Main being our main program and TranslateHtml our class that serves as a task for our workflow.
# Main class code
So we have a class, quite simple. It must be callable and return an array of results. I was, of course, inspired by the Translation Task class to see how the code should be organized. Let's remember, the idea is to do a custom task that should be able to chain with others that we could use in a workflow. Note that it is even possible to create workflows of workflows. This can be quite useful if you want to organize your tasks/workflows in the form of well-designed bricks.
# Workflow configuration
Notice our workflow. We could chain several in a row in order to chain a list of complex treatments.
We launch our unicorn server:
# Server start command
export CUDA_VISIBLE_DEVICES="" & uvicorn main:app --reload I put CUDA_VISIBLE_DEVICES empty to say "do not use my GPU".
Let's verify the processing of our API:
It is worth noting that the reference translate task used in our TranslateHtml Conversion Task automatically chooses the most appropriate model for your translation! Also, if I change my call and I put "it", my first request will have to wait for the download of the appropriate model.
Keep in mind that other specialized models will do a better job of translation. GPT-4 in particular, but other services made for this task will be more suitable. And in any case, it will not replace a real translator and proofreading. However, these text treatments will be sufficient to classify your documents and query them.
So here's a first introduction to TxtAi, and we'll have the opportunity to talk about it very soon as it constitutes a superb alternative to LlamaIndex.