Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

Below, is the json structure I am pulling from my online weather station. I am also including a json_to_csv python script that is supposed to convert json data to csv output, but only returns a "Key" error. I want to pull data from "current_observation": only.

"response": { "features": { "conditions": 1 , "current_observation": { "display_location": { "latitude":"40.466442", "longitude":"-85.362709", "elevation":"280.4" "observation_time_rfc822":"Fri, 26 Jan 2018 09:40:16 -0500", "local_time_rfc822":"Sun, 28 Jan 2018 11:22:47 -0500", "local_epoch":"1517156567", "local_tz_short":"EST", "weather":"Clear", "temperature_string":"44.6 F (7.0 C)", import csv, json, sys inputFile = open("pywu.cache.json", 'r') #open json file outputFile = open("CurrentObs.csv", 'w') #load csv file data = json.load(inputFile) #load json content inputFile.close() #close the input file output = csv.writer(outputFile) #create a csv.write output.writerow(data[0].keys()) for row in data: output = csv.writer(outputFile) #create a csv.write output.writerow(data[0].keys()) for row in data: output.writerow(row.values()) #values row

What's the best method to retrieve the temperature string and convert to .csv format? Thank you!

So the output you want is a CSV file with rows of just a single column, each showing only temperature_string? – NFB Feb 1, 2018 at 17:40

import pandas as pd
df = pd.read_json("pywu.cache.json")
df = df.loc[["local_time_rfc822", "weather", "temperature_string"],"current_observation"].T
df.to_csv("pywu.cache.csv")
maybe pandas can be of help for you. the .read_json() function creates a nice dataframe, from which you can easily choose the desired rows and columns. and it can save as csv as well.
to add latitude and longitude to the csv-line, you can do this:
df = pd.read_json("pywu.cache.csv")
df = df.loc[["local_time_rfc822", "weather", "temperature_string", "display_location"],"current_observation"].T
df = df.append(pd.Series([df["display_location"]["latitude"], df["display_location"]["longitude"]], index=["latitude", "longitude"]))
df = df.drop("display_location")
df.to_csv("pywu.cache.csv")
to print the location in numeric values, you can do this:
df = pd.to_numeric(df, errors="ignore")
print(df['latitude'], df['longitude'])
                How would you go about parsing latitude/ longitude using pandas and writing a single row output in CSV format?
– user5834454
                Feb 5, 2018 at 15:17
                not sure, what you mean with parsing exactly. adding latitutude and longitude can be done like this for example (i will add it to the answer)
– habet
                Feb 6, 2018 at 16:28
                Thanks for amending the code.  By parsing, I mean extracting both latitude and longitude from the string "display_location".  Not sure if it's possible to just display the numerical value for each?
– user5834454
                Feb 6, 2018 at 16:51
                sure, you can convert the resulting pd.Series() object with pd.to_numeric to a numeric values. with errors="ignore" you skip the non-numeric values. Then you can index the object just like a dict. (see code in answer). and sorry for all the editing.
– habet
                Feb 6, 2018 at 17:20
                NP with all the edits - it helps better understand were code is placed, within pandas.  With your most recent additions, do I add pd.to_numeric directly after the pd.Series and before df.drop?
– user5834454
                Feb 6, 2018 at 18:02
This will find all keys (e.g. "temperature_string") specified inside of the json blob and then write them to a csv file. You can modify this code to get multiple keys.
import csv, json, sys
def find_deep_value(d, key):
# Find a the value of keys hidden within a dict[dict[...]]
# Modified from https://stackoverflow.com/questions/9807634/find-all-occurrences-of-a-key-in-nested-python-dictionaries-and-lists
# @param d dictionary to search through
# @param key to find
    if key in d:
        yield d[key]
    for k in d.keys():
        if isinstance(d[k], dict):
            for j in find_deep_value(d[k], key):
                yield j
inputFile = open("pywu.cache.json", 'r')  # open json file
outputFile = open("mypws.csv", 'w')  # load csv file
data = json.load(inputFile)  # load json content
inputFile.close()  # close the input file
output = csv.writer(outputFile)  # create a csv.write
# Gives you a list of temperature_strings from within the json
temps = list(find_deep_value(data, "temperature_string"))
output.writerow(temps)
outputFile.close()
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.