Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

So this code works pretty good I get first desired output to the https://huggingface.co/chat/conversation/ :

from requests.sessions import Session
from json import loads
prompt = "Explain first condition in english?"
session = Session()
session.get(url="https://huggingface.co/chat/")
res = session.post(url="https://huggingface.co/chat/conversation")
assert res.status_code == 200, "Failed to create new conversation"
conversation_id = res.json()["conversationId"]
url = f"https://huggingface.co/chat/conversation/{conversation_id}"
max_tokens = int(2000) - len(prompt)
if max_tokens > 1904:
    max_tokens = 1904
res = session.post(
    url=url,
    json={
        "inputs": prompt,
        "parameters": {
            "temperature": 0.5,
            "top_p": 0.95,
            "repetition_penalty": 1.2,
            "top_k": 50,
            "truncate": 1024,
            "watermark": False,
            "max_new_tokens": max_tokens,
            "stop": ["<|endoftext|>"],
            "return_full_text": False,
        "stream": False,
        "options": {"use_cache": False},
    stream=False,
    data = res.json()
except ValueError:
    print("Invalid JSON response")
    data = {}
data = data[0] if data else {}
data.get("generated_text", "")
And it returns such output:
'Sure! The first condition you mentioned ....  Is there something specific you would like me to explain about this condition?'
However I dont know how to send second request?
Next code completely  ruins the chat:
res = session.post(
    url=url,
    json={
        "inputs": "GIVE more information",
        "parameters": {
            "temperature": 0.5,
            "top_p": 0.95,
            "repetition_penalty": 1.2,
            "top_k": 50,
            "truncate": 1024,
            "watermark": False,
            "max_new_tokens": max_tokens,
            "stop": ["<|endoftext|>"],
            "return_full_text": False,
        "stream": False,
        "options": {"use_cache": False},
    stream=False,
    data = res.json()
except ValueError:
    print("Invalid JSON response")
    data = {}
data.get("generated_text", "")
Output:
{'error': 'Model is overloaded', 'error_type': 'overloaded'}
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.