cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

LLM, RAG and Cloud Foundry: No space left on device

Go to solution
marcus_schiffer
Active Participant

on ‎2024 Apr 24 10:48 AM

0 Kudos
1,839

Hi,

 

we were following a set of blogs and tutorials like:

https://community.sap.com/t5/technology-blogs-by-sap/enhancing-s-4hana-with-sap-hana-cloud-vector-st...

https://developers.sap.com/tutorials/ai-core-genai-hana-vector.html

 

We need to use the features in a cloud foundry python app. Unfortunately the installation of

package 

sentence-transformers (as part of the requirements.txt file) always leads to a "no space left on device" error during build.
The memory limit is set to 8G, the disk space to 10G in the manifest.yml.
 
How can we solve this issue in the SAP BTP cloud foundry environment ?
 
Any help is highly appreciated.
 
Regards
Marcus 

 

Know the answer?

Help others by sharing your knowledge.

Answer

Need more details?

Request clarification before answering.

Show replies
Show replies

Accepted Solutions (1)

Accepted Solutions (1)

Dinu
Active Contributor
‎2024 Apr 24 11:00 AM

Sometimes staging requires more disk space.  If you vendor your requirements, there would be no need for this.  Make sure to target platform as cflinuxfs4(Ubuntu 22.04). 

See a similar issue: https://community.sap.com/t5/technology-q-a/could-not-install-packages-due-to-an-environmenterror-er...

 

Show replies
Show replies
marcus_schiffer
Active Participant
‎2024 Apr 24 11:43 AM
0 Kudos
Thanks for the information. We get one step further but now we see 7.50 MiB / 2.73 GiB [>-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 0.27%Error unmarshalling the following into a cloud controller error: <html> <head><title>413 Request Entity Too Large</title></head> <body> <center><h1>413 Request Entity Too Large</h1></center> <hr><center>nginx</center> </body> </html>
marcus_schiffer
Active Participant
‎2024 Apr 24 11:43 AM
0 Kudos
How could this be solved ?
Dinu
Active Contributor
‎2024 Apr 24 12:18 PM
0 Kudos
You need to provide some more context to this. When do you get this error? This appears to be unrelated to the original question. So it might be better to pose a new question to get fresh eyeballs to look at it without the cloud of the previous issue.
marcus_schiffer
Active Participant
‎2024 Apr 25 6:13 AM
The main problem indeed is the size of the app. The pip install of sentence-transform installs a lot of prerequisites. When choosing a proper combination of these and install them before installing it, it will reduce the size of the app to below the limit for storage capacity of the cf environment.
Dinu
Active Contributor
‎2024 Apr 25 2:10 PM
The error "no space left on device" usually is because the build tools uses /tmp or other file systems with limitted size. The disk size available for the app itself you can control with the quota for the app. When do you encounter the error "Error unmarshalling"? BTW, ML applications are normally not run in CF, but in AI Core. It provides much bigger machines needed for these.
marcus_schiffer
Active Participant
‎2024 Apr 26 6:08 AM
0 Kudos
Using AICore might be an alternative. However, in our case we already have all data in a HANA cloud as part of a cloud foundry tenant/space architecture. Also the machine learning part is only regression and time series forecast and data volume is limited. So the HANA ML functions in our existing landscape are sufficient. For the RAG the HANA also offers the vector store so that everything is on board for a ML/chatGPT/ application.
Dinu
Active Contributor
‎2024 Apr 26 6:49 PM
0 Kudos
@marcus_schiffer, Which packages were you installing? I could install sentence-transformers=2.7.0 and langchain=0.1.16 without vendoring in 4GB RAM, 10GB disk. The model (all-MiniLM-L6-v2) was downloaded at runtime and used. I was considering installing the packages also at runtime, but there was no need.
marcus_schiffer
Active Participant
‎2024 May 08 6:56 AM
0 Kudos
I installed via requirements:
marcus_schiffer
Active Participant
‎2024 May 08 6:56 AM
0 Kudos
I installed via requirements:langchain langchain_community transformers==4.34.0 tqdm -f https://download.pytorch.org/whl/torch_stable.html torch== 2.2.2+cpu scipy huggingface-hub==0.16.4 Pillow sentence_transformers
Dinu
Active Contributor
‎2024 May 13 6:20 PM
0 Kudos

I have all these components albeit different versions:

- python=3.11.6=hab00c5b_0_cpython
- langchain=0.1.16=pyhd8ed1ab_0
- langchain-community=0.0.34=pyhd8ed1ab_0
- pytorch=2.1.2=cpu_generic_py311h1584bb0_3
- huggingface_hub=0.22.2=pyhd8ed1ab_0
- pillow=10.3.0=py311h18e6fac_0
- scipy=1.13.0=py311h64a7726_0
- sentence-transformers=2.7.0=pyhd8ed1ab_0
- tqdm=4.66.2=pyhd8ed1ab_0

I could push the app without vendoring using -m 4G -k 10G

So, it seems doable. If this is a viable, what could be sustainable throughput, is a different question. 

Answers (0)

Ask a Question