Discussion on "APEX in the AI era"

Plamen Mushkov · 2024-06-21T09:12:58.812Z

What's new in APEX 24.1 This version of APEX is the first major one that brings AI services closer to developers. In the heart of them all are the Large language Models (LLM). The new Generative AI Services Workspace Utility allows developers to conf...

I get error: ORA-00001: unique constraint (APEX_240200.WWV_REMOTE_SRVR_AI_BLD_UK) violated

do you have any idea? how to resolve

Hi Plamen,

Good Day and congrats for such an enriching blog.

Everything worked as your blog says (at least the very first time) and then onwards ngrok started giving troubles; may be because of the free account they might limit the traffic etc., but no issues.

I have created an Oracle Linux instance in the OCI Tenancy (because my APEX app is there) and completed the OLLAMA setup successfully. Checked that the ollama service is getting accessed through internet; checked that I could access the OLLMA service over internet using postman as well as curl ang got results correctly.

However, when tried to use the same URL (with the bare public IP that worked with cURL and Postman) APEX errored out saying taht the requested URL is prohibited.

I created an API Gateway in OCI and trying to pass through it to reach the OLLAM service, however the AP Gateway is throwing error.

If you are interested in probing and tweaking in to this architecture (accessing OLLAMA securely via OCI API Gateway) I am open to mutual colaboration.

Regards

Hi There. Don't bother...I got it working..!! Thanks for your post though, I got the idea of using open source local LLMs with APEX using my own OCI Instance and OCI API Gateways.

Regards

Hi Ravi, I'm glad it worked for you. As you have discovered, Ngrok's free accounts have some limitations - they use the default URLs which are too long. It also generates a new URL every time you re-start the service, so you need to re-configure the services using it.

But since you have Ollama on OCI and Ngrok is out of the equation, it's all good. Hope it performs well on the OCI and the cost is not too high.

Hi there

Thanks for your reply. For just a lab environment its doing "ok-ishh".

The VM instance I used is the smallest one (1 CPU + 16GB RAM) in OCI. After trying to pull more than one model with ollama, I found out that even the default disk size is also not enough and I had to increase that. :-)

During my testing phase there were some hiccups because I monitored that the ollama was thriving for both CPU and RAM and meanwhile the OCI API Gateway gets timed out.

So, for a real world scenario, one must go with a GPU based instance, a baremetal instance or at least an instance with considerable amout of resources so that the things work consistently.

My next adventure would be adding the RAG for this architecture...I might post the whole thing once I get things working.

Regards.

Ravi

Hello, This is an awesome post. I tried it but stuck in the Ngrok part. The Ngrok base URL is not running in Postman. getting 403 - forbidden error. Can we run the Base URL of Ngrok in POSTMAN? In the APEX part, I configured it and then created a button to invoke the Chatbox. Invoking is working fine but when we enter a Prompt then it gives this error "Generative Al Service with given ID or static ID not found."

The Ngrok setup first:

Try adding the host_header when starting Ngrok: ngrok http 127.0.0.1 --host-header '127.0.0.1:11434'

After starting the tunnel, open the Ngrok URL in the browser and when displayed a default page, click on "Visit Site" button. It only need to be done once.

If the above doesn't work, you can try to run it using a .yml file - here is the documentation (ngrok.com/docs/guides/site-to-site-apis-mtls).

The file will look like this:

version: 2
authtoken: YOUR-AUTH-TOKEN
tunnels:
  ollama:
    proto: http
    addr: http://127.0.0.1:11434/
    host_header: 127.0.0.1:11434

Pay attention to the host_header part - it should fix your 403 error.

As to Ollama, run ollama list in the Terminal. You will get a list of your models. You can get the Model name from there and use it in your APEX AI Services configuration. This should resolve your second problem. When using the newly configured AI Service for the AI Assistant component for example, it is a drop down menu, you can't enter a wrong name.

Discussion

APEX in the AI era

Responses(3)

Recent in Forum

Search Hashnode

APEX in the AI era

Responses(3)

Recent in Forum