FastAPI 0.65.2 POST request fails with "value is not a valid dict" when using the Requests library; 0.65.1 works (with a caveat)
First off, thank you for creating this marvellous piece of software. It is a real life-changer.
I hit a very odd bug while implementing some unit tests. Using FastAPI
0.65.2
, a POST request via the
requests
module (
requests.post
) consistently returns the following error:
from fastapi import FastAPI
from pydantic import BaseModel
import uvicorn
app = FastAPI()
class Data(BaseModel):
field: str
@app.post("/test")
def test_data(data: Data):
return "Polo"
if __name__=='__main__':
uvicorn.run(app)
And the requests counterpart:
import requests as rq
def test():
result = rq.post('http://127.0.0.1:8000/test', data={"field": "Marco"})
print(f"==[ result: {result.json()}")
if __name__=="__main__":
test()
The result is
==[ result: {'detail': [{'loc': ['body'], 'msg': 'value is not a valid dict', 'type': 'type_error.dict'}]}
If I downgrade FastAPI to 0.65.1, I still get an error, but a different one:
==[ result: {'detail': [{'loc': ['body', 0], 'msg': 'Expecting value: line 1 column 1 (char 0)', 'type': 'value_error.jsondecode', 'ctx': {'msg': 'Expecting value', 'doc': 'field=heya', 'pos': 0, 'lineno': 1, 'colno': 1}}]}
This can be solved by JSONifying the dictionary:
import requests as rq
import json as js
def test():
result = rq.post('http://127.0.0.1:8000/test', data = js.dumps({"field": "Marco"}))
print(f"==[ result: {result.json()}")
if __name__=="__main__":
test()
Running the above prints
==[ result: Polo
I am slightly perplexed because I am not sure if it is the requests library that is to blame or FastAPI. POSTing via ReDoc works without hiccups with both versions. I am at a loss as to why the second parameter to requests.post is accepted as JSON when it should actually be a dictionary.
Please let me know if I'm missing something obvious or whether I should redirect this to the maintainers of requests. Thank you again for making the lives of developers so much easier.
Cheers!
deepak-sadulla, gabrielronai, ettzzz, mezhaka, mojixcoder, karlkovaciny, dmckee-ebsco, sean-adler, ebubekir, bsamseth, and 39 more reacted with thumbs up emoji
merc1er, mattbesancon, SeeObjective, bbusa, bgalvao, and cisaacstern reacted with hooray emoji
storca, montaro, merc1er, SeeObjective, sahil-webner, bbusa, cisaacstern, and artemonsh reacted with heart emoji
All reactions
@Mause
Thank you for the quick reply. What you suggested indeed fixed the problem, so I will mark this as resolved, but I am still perplexed. I never knew that the data parameter was for form data - I have always used data until I encountered this issue. This is from the requests documentation:
Requests’ simple API means that all forms of HTTP request are as obvious. For example, this is how you make an HTTP POST request:
>>> r = requests.post('https://httpbin.org/post', data = {'key':'value'})
Also, I'm not sure why downgrading FastAPI works (subject to the JSONification caveat above).
Still, all is well that ends well. :) Thank you for your help!
deepak-sadulla, revanthparepalli, crisconru, madprogramer, and GautamBose reacted with thumbs up emoji
Ran4 reacted with thumbs down emoji
All reactions
changed the title
FastAPI 0.65.2 POST request fails with "value is not a valid dict" when using Requests library; 0.65.1 works (with a caveat)
FastAPI 0.65.2 POST request fails with "value is not a valid dict" when using the Requests library; 0.65.1 works (with a caveat)
Jun 15, 2021
I just tried v0.67.0 and I'm still seeing the issue. Taking the initial example, with the following request:
curl -X POST --header "Content-Type: text/plain" --data "{\"field\": \"Marco\"}" http://127.0.0.1:8000/test
This doesn't work (shows INFO: 127.0.0.1:35400 - "POST /test HTTP/1.1" 422 Unprocessable Entity):
from fastapi import FastAPI
from pydantic import BaseModel
import uvicorn
app = FastAPI()
class Data(BaseModel):
field: str
@app.post("/test")
async def test_data(data: Data):
print(data)
return "Polo"
if __name__=='__main__':
uvicorn.run(app)And this works fine (prints {'field': 'Marco'}):
from fastapi import FastAPI
import uvicorn
from starlette.requests import Request
app = FastAPI()
@app.post("/test")
async def test_data(request: Request):
print(await request.json())
return "Polo"
if __name__=='__main__':
uvicorn.run(app)I got this error too. My use case was sending a base64 encoded image.
After trying a mix of the solutions above, I ended up with the below.
json_response = requests.post(
"http://host:port/abc",
json=data,
headers={"Content-Type": "application/json; charset=utf-8"},
Weirdly enough, Postman works despite not setting the charset.
yongkangc, Ostap1807, rybkinn, Tou7and, VinayChaudhari1996, and YannickLeRoux reacted with thumbs up emoji
sheeceg reacted with hooray emoji
sheeceg reacted with rocket emoji
All reactions
While try this example with GNS3 branch 3.0 on Ubuntu 20.04 LTS, the API call
```$ curl http://172.17.46.114:3080/v3/users/authenticate -d '{"username": "admin", "password": "admin"}' ``` fails with ```{"detail":[{"loc":["body"],"msg":"value is not a valid dict","type":"type_error.dict"}]}```.
The additional parameter appended to curl fixes this ```{"access_token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTYzMTI4NTc4NX0.kT1dAN4v1vHMTBJO2UaI2I7yAFhnSpBU9iqmdDuwtAQ","token_type":"bearer"} ```
Found after googling for the error above in
FastAPI 0.65.2 POST request fails with "value is not a valid dict" when using the Requests library; 0.65.1 works (with a caveat) #3373
tiangolo/fastapi#3373 (comment)
AWS SNS HTTP/HTTPS notification JSON format messages specify content-type of text/plain; charset=UTF-8. This causes the current version of FastAPI (0.68.1) to fail with a 422 exception discussed above, which appears to be a value is not a valid dict (type=type_error.dict) 400 exception when overriding the RequestValidationError and logging the exc as a string. Downgrading to 0.65.1 resolved the issue.
Example AWS SNS JSON message from
https://docs.aws.amazon.com/sns/latest/dg/sns-message-and-json-formats.html#http-notification-json
POST / HTTP/1.1
x-amz-sns-message-type: SubscriptionConfirmation
x-amz-sns-message-id: 165545c9-2a5c-472c-8df2-7ff2be2b3b1b
x-amz-sns-topic-arn: arn:aws:sns:us-west-2:123456789012:MyTopic
Content-Length: 1336
Content-Type: text/plain; charset=UTF-8
Host: myhost.example.com
Connection: Keep-Alive
User-Agent: Amazon Simple Notification Service Agent
"Type" : "SubscriptionConfirmation",
"MessageId" : "165545c9-2a5c-472c-8df2-7ff2be2b3b1b",
"Token" : "2336412f37...",
"TopicArn" : "arn:aws:sns:us-west-2:123456789012:MyTopic",
"Message" : "You have chosen to subscribe to the topic arn:aws:sns:us-west-2:123456789012:MyTopic.\nTo confirm the subscription, visit the SubscribeURL included in this message.",
"SubscribeURL" : "https://sns.us-west-2.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:us-west-2:123456789012:MyTopic&Token=2336412f37...",
"Timestamp" : "2012-04-26T20:45:04.751Z",
"SignatureVersion" : "1",
"Signature" : "EXAMPLEpH+DcEwjAPg8O9mY8dReBSwksfg2S7WKQcikcNKWLQjwu6A4VbeS0QHVCkhRS7fUQvi2egU3N858fiTDN6bkkOxYDVrY0Ad8L10Hs3zH81mtnPk5uvvolIC1CXGu43obcgFxeL3khZl8IKvO61GWB6jI9b5+gLPoBc1Q=",
"SigningCertURL" : "https://sns.us-west-2.amazonaws.com/SimpleNotificationService-f3ecfb7224c7233fe7bb5f59f96de52f.pem"
@Mause IMHO if the data arg is for FormData objects, then the post function help text needs to be updated. It currently reads: "param data: (optional) Dictionary, list of tuples, bytes, or file-like object to send..." But if you pass it a dictionary, you get back a status_code of 422. And thanks!
I think this issue is critical, I saw this on 0.70.0 too, make a POST request from httpx with json=msg, headers={"Content-Type": "application/json; charset=utf-8"} does not resolve this issue.
my pydantic model just simple like this:
class history_model(BaseModel):
process_order: int
process_name: str
request_dttm: str
request_body: Optional[Dict] = None
response_body: Optional[Dict] = None
class worker_model(BaseModel):
msg_no : str
topic_name : str
next_stage : str
entity_info : Dict
status_history : List[history_model]
track_change : Optional[List[Dict]] = None
when I test the json string in /docs, it work, but post it via httpx/requests it will return error:
{'detail': [{'loc': ['body'], 'msg': 'value is not a valid dict', 'type': 'type_error.dict'}]}
New update:
found the bug, when setup with pydantic model, fastapi expect request_body to be received as dict, not string, if you send request and feed the json string into json param like this:
requests.post(url=url, json=msg_json_string)
it will generate error {'detail': [{'loc': ['body'], 'msg': 'value is not a valid dict', 'type': 'type_error.dict'}]}
my FastAPI version is 0.70.0. if you setup the path operation with pydantic model, remember to send request as dict, not json, like this:
requests.post(url=url, json=json.loads(msg_json_string))
@tiangolo is this normal behavior ?
The reason the downgrade fixed it was due to the json post handling in the latest version - it was changed so it only tried to parse bodies with a valid Content-Type header as json, which the data parameter wouldn't set
This is a rather major change of behaviour!
It broke plenty of existing integrations at $work :(
I also have seen similar behavior and was able to work around it by adding validator to my models/schemas. Not sure it is the same issue, but maybe this helps
@classmethod
def get_validators(cls):
yield cls.validate_to_json
@classmethod
def validate_to_json(cls, value):
if isinstance(value, str):
return cls(**json.loads(value))
return value
I'm facing the same issue using fastapi 0.73.0 and have tried every solution posted in this thread with no success.
df_json = dataframe.to_json(orient='records')
prediction = requests.post('http://backend:8080/prediction/', json=json.loads(df_json), headers={"Content-Type": "application/json"})
class Userdata(BaseModel):
RPPA_HSPA1A : float
RPPA_XIAP : float
RPPA_CASP7 : float
RPPA_ERBB3 :float
RPPA_SMAD1 : float
RPPA_SYK : float
RPPA_STAT5A : float
RPPA_CD20 : float
RPPA_AKT1_Akt :float
RPPA_BAD : float
RPPA_PARP1 : float
RPPA_MSH2 : float
RPPA_MSH6 : float
RPPA_ACACA : float
RPPA_COL6A1 : float
RPPA_PTCH1 : float
RPPA_AKT1 : float
RPPA_CDKN1B : float
RPPA_GATA3 : float
RPPA_MAPT : float
RPPA_TGM2 : float
RPPA_CCNE1 : float
RPPA_INPP4B : float
RPPA_ACACA_ACC1 : float
RPPA_RPS6 : float
RPPA_VASP : float
RPPA_CDH1 : float
RPPA_EIF4EBP1 : float
RPPA_CTNNB1 : float
RPPA_XBP1 : float
RPPA_EIF4EBP1_4E : float
RPPA_PCNA : float
RPPA_SRC : float
RPPA_TP53BP1 : float
RPPA_MAP2K1 : float
RPPA_RAF1 : float
RPPA_MET : float
RPPA_TP53 : float
RPPA_YAP1 : float
RPPA_MAPK8 : float
RPPA_CDKN1B_p27 : float
RPPA_FRAP1 : float
RPPA_RAD50 : float
RPPA_CCNE2 : float
RPPA_SNAI2 : float
RPPA_PRKCA_PKC : float
RPPA_PGR : float
RPPA_ASNS : float
RPPA_BID : float
RPPA_CHEK2 : float
RPPA_BCL2L1 : float
RPPA_RPS6 : float
RPPA_EGFR : float
RPPA_PIK3CA : float
RPPA_BCL2L11 : float
RPPA_GSK3A : float
RPPA_DVL3 : float
RPPA_CCND1 : float
RPPA_RAB11A : float
RPPA_SRC_Src_pY416 :float
RPPA_BCL2L111 : float
RPPA_ATM : float
RPPA_NOTCH1 : float
RPPA_C12ORF5 : float
RPPA_MAPK9 : float
RPPA_FN1 : float
RPPA_GSK3A_GSK3B : float
RPPA_CDKN1B_p27_pT198 : float
RPPA_MAP2K1_MEK1 : float
RPPA_CASP8 : float
RPPA_PAI : float
RPPA_CHEK1 : float
RPPA_STK11 : float
RPPA_AKT1S1 : float
RPPA_WWTR1 : float
RPPA_CDKN1A : float
RPPA_KDR : float
RPPA_CHEK2_2 : float
RPPA_EGFR_pY1173 : float
RPPA_EGFR_pY992 : float
RPPA_IGF1R : float
RPPA_YWHAE : float
RPPA_RPS6KA1 : float
RPPA_TSC2 : float
RPPA_CDC2 : float
RPPA_EEF2 : float
RPPA_NCOA3 : float
RPPA_FRAP1 : float
RPPA_AR : float
RPPA_GAB2 : float
RPPA_YBX1 : float
RPPA_ESR1 : float
RPPA_RAD51 : float
RPPA_SMAD4 : float
RPPA_CDH3 : float
RPPA_CDH2 : float
RPPA_FOXO3 : float
RPPA_ERBB2_HER : float
RPPA_BECN1 : float
RPPA_CASP9 : float
RPPA_SETD2 : float
RPPA_SRC_Src_mv : float
RPPA_GSK3A_alpha : float
RPPA_YAP1_pS127 : float
RPPA_PRKCA_alpha : float
RPPA_PRKAA1 : float
RPPA_RAF1_pS338 : float
RPPA_MYC : float
RPPA_PRKAA1_AMPK : float
RPPA_ERRFI1_MIG : float
RPPA_EIF4EBP1_2 : float
RPPA_STAT3 : float
RPPA_AKT1_AKT2_AKT3 : float
RPPA_NF2 : float
RPPA_PECAM1 : float
RPPA_BAK1 : float
RPPA_IRS1 : float
RPPA_PTK2 : float
RPPA_ERBB3_2 : float
RPPA_FOXO3_a : float
RPPA_RB1_Rb : float
RPPA_MAPK14_p38 : float
RPPA_NFKB1 : float
RPPA_CHEK1_Chk1 : float
RPPA_LCK : float
RPPA_XRCC5 : float
RPPA_PARK7 : float
RPPA_DIABLO : float
RPPA_CTNNA1 : float
RPPA_ESR1_ER : float
RPPA_IGFBP2 : float
RPPA_STMN1 : float
RPPA_WWTR1_TAZ : float
RPPA_CASP3 : float
RPPA_JUN : float
RPPA_CCNB1 : float
RPPA_CLDN7 : float
RPPA_PXN : float
RPPA_RPS6KB1_p : float
RPPA_KIT : float
RPPA_CAV1 : float
RPPA_PTEN : float
RPPA_BAX : float
RPPA_SMAD3 : float
RPPA_ERBB2 : float
RPPA_MET_c : float
RPPA_ERCC1 : float
RPPA_MAPK14 : float
RPPA_BIRC2 : float
RPPA_PIK3R1 : float
RPPA_BCL2 : float
RPPA_PEA : float
RPPA_EEF2K : float
RPPA_RPS6KB1_p70 : float
RPPA_MRE11A : float
RPPA_KRAS : float
RPPA_ARID1A : float
RPPA_YBX1_yb : float
RPPA_NOTCH3 : float
RPPA_EIF4EBP1_3 : float
RPPA_XRCC1 : float
RPPA_ANXA1 : float
RPPA_CD49 : float
RPPA_SHC1 : float
RPPA_PDK1 : float
RPPA_EIF4E : float
RPPA_MAPK1_MAPK3 : float
RPPA_PTGS2 : float
RPPA_PRKCA : float
RPPA_EGFR_egfr : float
RPPA_RAB25 : float
RPPA_RB1 : float
RPPA_MAPK1 : float
RPPA_TFF1 : float
class config:
orm_mode = True
@app.post("/prediction/")
async def create_item(userdata: Userdata):
df = pd.DataFrame(userdata)
y = model.predict(df)
y = [0 if val < 0.5 else 1 for val in y]
if y == 1:
survival = 'You will survive.'
if y == 0:
survival = 'You will not survive.'
return {'Prediction': survival}
Userdata.txt
Attached is the dna data text file being used. Any help would be appreciated.
@ahutchful
The error can be a bit confusing, but here the endpoint is expecting the request body to be a dictionary.
However, your request body is an array of records.
This is explained here https://fastapi.tiangolo.com/tutorial/body-multiple-params/
I found the explanation confusing as other people have it seems.
Essentially, if you have a single body parameter, then FastAPI expects that item as the actual body of the request.
Whereas if you have multiple parameters it expects the body to be a dict with the parameter names as keys.
You can get around this though
But if you want it to expect a JSON with a key item and inside of it the model contents, as it does when you declare extra body parameters, you can use the special Body parameter embed:
item: Item = Body(..., embed=True)
It does this parsing automatically and the error can be misleading if you don't understand how the parsing works.
I just updated from 0.63 to 0.78 and I had the error too.
I fixed it by changing the frontend from
const response = await fetch(url, {
method: 'POST',
body: JSON.stringify(body),
const response = await fetch(url, {
method: 'POST',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
I Solved It With The "Content-type" Parameter In Requests Library. Attaching My code for reference.
response = requests.post(url, json, 'application/dict')
class Query(BaseModel): query: dict
@app.post('/query/') def get_records(query: Query): print("aaaa=-===", query) return {"records": "a"}
Swift Alamofire encoder: JSONParameterEncoder.prettyPrinted is working
let url = "http://192.168.31.83:8000/create_user"
AF.request(url, method: .post, parameters: parameters, encoder: JSONParameterEncoder.prettyPrinted).response { response in
guard let data = response.data else { return }
let json = try? JSONSerialization.jsonObject(with: data, options: .fragmentsAllowed) as? [String : Any]
print(json)
let user = User(fromJson: JSON.init(json as Any))
print(user)
Thanks for the help @Mause! 🍰
And thanks for the discussion everyone. ☕
That change was a fix for a vulnerability. That's why it can't just assume and read JSON, and that's why it has to be there. You can read more in the release notes: https://fastapi.tiangolo.com/release-notes/#0652
Sorry for the long delay! 🙈 I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.
Also, using requests.post("/create-user", json={"some": "data"}) requires the value passed to json to be the Python objects because Requests serializes that to JSON bytes (a string). If you serialize it before and pass it a string, then Requests will send it as a JSON string data type, not the pure data serialized as JSON.
It's not related to FastAPI, FastAPI expects standard JSON. It's related to how Requests expects the data in the parameter and how it serializes automatically for you.
