Problem to execute Examples in "Solving Your Business Problems Using Prompts...

[Nobue]

Hello,

I am searching for any help to solve a problem on a content in the learning journey, "Solving Your Business Problems Using Prompts and LLMs in SAP's Generative AI Hub".

Chapter: Leveraging the Power of LLMs Using SDK for Generative AI Hub

Section: Identifying the Need for Using Generative-AI-Hub-SDK

I set up config.json and try to execute examples there but I encounter errors and cannot find a solution to them.

Is there anyone help me?

One example:

--------------

from gen_ai_hub.proxy.native.openai import completions
response = completions.create(
model_name="tiiuae--falcon-40b-instruct",
prompt="The Answer to the Ultimate Question of Life, the Universe, and Everything is",
max_tokens=7,
temperature=0
)
print(response)

--------------

Error I get

-------------

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[35], line 2
      1 from gen_ai_hub.proxy.native.openai import completions
----> 2 response = completions.create(
      3     model_name="tiiuae--falcon-40b-instruct",
      4     prompt="The Answer to the Ultimate Question of Life, the Universe, and Everything is",
      5     max_tokens=7,
      6     temperature=0
      7 )
      8 print(response)

File ~\AppData\Local\Programs\Python\Python313\Lib\site-packages\gen_ai_hub\proxy\native\openai\clients.py:186, in Completions.create(self, prompt, model, deployment_id, model_name, config_id, config_name, **kwargs)
    179 model_name = if_set(model_name, if_set(model))
    180 model_identification = kwargs_if_set(
    181     deployment_id=deployment_id,
    182     model_name=model_name,
    183     config_id=config_id,
    184     config_name=config_name,
    185 )
--> 186 deployment = proxy_client.select_deployment(**model_identification)
    187 model_name = deployment.model_name or '???'
    188 with set_deployment(deployment):

File ~\AppData\Local\Programs\Python\Python313\Lib\site-packages\gen_ai_hub\proxy\gen_ai_hub_proxy\client.py:238, in GenAIHubProxyClient.select_deployment(self, raise_on_multiple, **search_key_value)
    236     return matched_deployments[0]
    237 else:
--> 238     raise ValueError('No deployment found with: ' + ', '.join(
    239         [f'deployment.{k} == {v}' for k, v in search_key_value.items()]
    240     ))

ValueError: No deployment found with: deployment.model_name == tiiuae--falcon-40b-instruct

--------------

[MikeJ · Accepted solution]

Based on the error message, the possibility here is that you may not have deployed the model.

It looks as if you may have missed implementing the following note mentioned at the end of the Unit 1 Lesson 1:

Note

You can try using any other model in the code that you have deployed during the setup. For example, in case you have deployed the latest a mistralai model you can use that model instead of "tiiuae—falcon-40b-instruct" in this code.

best regards

[Yogananda]

Hi @Nobue 

As error is clearly mentioned "tiiuae--falcon-40b-instruct" deployment doesn't exist.

you will have to try another AI model which is deployed and update it in your python code

[Nobue]

Thank you for your response. It is part of a Learning Journey. The code is provided in the training but there is no guidance how to set the deployment itself to be available for the python code. the content right before the code is only how to set up config.json for SDK.