After upgrading to the latest streamlit version now when i run the function it display the below error:
ArrowInvalid: (‘Could not convert int64 with type numpy.dtype: did not recognize Python value type when inferring an Arrow data type’, ‘Conversion failed for column Data Type with type object’)
Traceback:
File “F:\AIenv\lib\site-packages\streamlit\script_runner.py”, line 350, in _run_script
exec(code, module.
dict
)
File “f:\AIenv\streamlit\app2.py”, line 1051, in
main()
File “f:\AIenv\streamlit\app2.py”, line 597, in main
explore(df)
File “f:\AIenv\streamlit\app2.py”, line 91, in explore
st.write(df_types)
File “F:\AIenv\lib\site-packages\streamlit\elements\write.py”, line 182, in write
self.dg.dataframe(arg)
File “F:\AIenv\lib\site-packages\streamlit\elements\dataframe_selector.py”, line 85, in dataframe
return self.dg._arrow_dataframe(data, width, height)
File “F:\AIenv\lib\site-packages\streamlit\elements\arrow.py”, line 82, in
arrow_dataframe
marshall(proto, data, default_uuid)
File “F:\AIenv\lib\site-packages\streamlit\elements\arrow.py”, line 160, in marshall
proto.data = type_util.data_frame_to_bytes(df)
File “F:\AIenv\lib\site-packages\streamlit\type_util.py”, line 371, in data_frame_to_bytes
table = pa.Table.from_pandas(df)
File “pyarrow\table.pxi”, line 1479, in pyarrow.lib.Table.from_pandas
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 591, in dataframe_to_arrays
for c, f in zip(columns_to_convert, convert_fields)]
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 590, in
arrays = [convert_column(c, f)
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 577, in convert_column
raise e
File “F:\AIenv\lib\site-packages\pyarrow\pandas_compat.py”, line 571, in convert_column
result = pa.array(col, type=type
, from_pandas=True, safe=safe)
File “pyarrow\array.pxi”, line 301, in pyarrow.lib.array
File “pyarrow\array.pxi”, line 83, in pyarrow.lib._ndarray_to_array
File “pyarrow\error.pxi”, line 84, in pyarrow.lib.check_status
import streamlit as st
import pandas as pd
def explore(df):
# DATA
st.write('Data:')
st.write(df)
# SUMMARY
df_types = pd.DataFrame(df.dtypes, columns=['Data Type'])
numerical_cols = df_types[~df_types['Data Type'].isin(['object',
'bool'])].index.values
df_types['Count'] = df.count()
df_types['Null Values'] = df.isnull().sum()
df_types['Unique Values'] = df.nunique()
df_types['Min'] = df[numerical_cols].min()
df_types['Max'] = df[numerical_cols].max()
df_types['Average'] = df[numerical_cols].mean()
df_types['Median'] = df[numerical_cols].median()
df_types['St. Dev.'] = df[numerical_cols].std()
st.write('Summary:')
st.write(df_types)
#show Summary
if st.checkbox("Show Summary"):
explore(df)
what this error mean ? and How to fix this error
I had the same problem. It’s a bug that came with streamlit 0.85.0. I hope developers will solve it soon.
The current solution is downgrade to version 0.84.2
pip install streamlit==0.84.2
Me too. Very simple read in of xlsx file with diverse column data types.
st.markdown("### Books In Print")
df = pd.read_excel("BIP4streamlit.xlsx")
df['title'].astype(str)
df['Contributor 1'].astype(str)
File "/Users/fred/.virtualenvs/nimbleAI/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 571, in convert_column
result = pa.array(col, type=type_, from_pandas=True, safe=safe)
File "pyarrow/array.pxi", line 301, in pyarrow.lib.array
File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array
File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: ('Could not convert A Day Book with Prompts with type str: tried to convert to int', 'Conversion failed for column title with type object')
upgraded to 0.88.
Hi all
A preferable solution for this is to use the old dataframe serializer by setting this in your .streamlit/config.toml
file:
[global]
dataFrameSerialization = "legacy"
This allows you to continue upgrading to the latest version of Streamlit and getting all the latest goodies!
More info about Arrow here.
Meanwhile, I’ll forward this thread to our eng team. We want to find all instances where Arrow isn’t working well and fix them
Hi there ,
After some investigation, it turns out that pyarrow
has an issue with numpy.dtype
values (which df.dtypes
returns).
The issue has been filed and hopefully will be taken care of soon.
In the meantime, a custom exception with a more useful error message will be thrown.
In some cases, a possible workaround is to convert DataFrame cells to strings with df.astype(str)
.
For example:
df = pd.DataFrame([["foo", "bar"]])
foo = df.dtypes.astype(str)
st.write(foo)
I hope this helps ease some of the burden.
Please let us know if that doesn’t work for you, and we’ll try to come up with a better solution.
And thanks for using Streamlit
Yes you’re right this is sucks because there are a lot of new features in the latest version that can be very useful.
Hope that this issue is being debugued by streamlit’s creators
Dear users,
If the workaround proposed above didn’t solve your issue, please use the approach described here.
This allows you to continue upgrading to the latest version of Streamlit and getting all the latest goodies!
Sorry for the inconvenience! We’re waiting for Arrow to fix this issue.
I tried to add this line is the config.toml but when i tried to run streamlit app it crash and display the below error:
Traceback (most recent call last):
File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "F:\AIenv\lib\site-packages\streamlit\__main__.py", line 23, in <module>
main()
File "F:\AIenv\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "F:\AIenv\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "F:\AIenv\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "F:\AIenv\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "F:\AIenv\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "F:\AIenv\lib\site-packages\streamlit\cli.py", line 161, in main_run
bootstrap.load_config_options(flag_options=kwargs)
File "F:\AIenv\lib\site-packages\streamlit\bootstrap.py", line 318, in load_config_options
config.get_config_options(force_reparse=True, options_from_flags=options_from_flags)
File "F:\AIenv\lib\site-packages\streamlit\config.py", line 1069, in get_config_options
_update_config_with_toml(file_contents, filename)
File "F:\AIenv\lib\site-packages\streamlit\config.py", line 946, in _update_config_with_toml
parsed_config_file = toml.loads(raw_toml)
File "F:\AIenv\lib\site-packages\toml\decoder.py", line 262, in loads
original, i)
toml.decoder.TomlDecodeError: Found invalid character in key name: '#'. Try quoting the key name. (line 1 column 3 char 2)
ns\ms-python.python-2020.9.114305\pythonFiles\lib\python\debugpy\launcher' '58862' '--' '-m' 'streamlit' 'run' 'f:\AIenv\streamlit\app2.py' '--server.port' '5678'
Traceback (most recent call last):
File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\lt gm\appdata\local\programs\python\python37\Lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "F:\AIenv\lib\site-packages\streamlit\__main__.py", line 23, in <module>
main()
File "F:\AIenv\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "F:\AIenv\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "F:\AIenv\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
return ctx.invoke(self.callback, **ctx.params)
File "F:\AIenv\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "F:\AIenv\lib\site-packages\streamlit\cli.py", line 161, in main_run
bootstrap.load_config_options(flag_options=kwargs)
File "F:\AIenv\lib\site-packages\streamlit\bootstrap.py", line 318, in load_config_options
config.get_config_options(force_reparse=True, options_from_flags=options_from_flags)
File "F:\AIenv\lib\site-packages\streamlit\config.py", line 1069, in get_config_options
_update_config_with_toml(file_contents, filename)
File "F:\AIenv\lib\site-packages\streamlit\config.py", line 946, in _update_config_with_toml
parsed_config_file = toml.loads(raw_toml)
File "F:\AIenv\lib\site-packages\toml\decoder.py", line 262, in loads
original, i)
toml.decoder.TomlDecodeError: Found invalid character in key name: '#'. Try quoting the key name. (line 1 column 3 char 2)
and I cannot run the app unless i delete the config.toml file
so it looks that i am doing something wrong or this solution is not working
To: @leb_dev
It looks like it can’t parse the config.toml
file.
Can you post the contents of the file? I’ll be happy to help you resolve it!
P.S. Or you can open an issue on GitHub if you like.
this is the content of the file without adding the line that you suggested before.
thank you for your time.
# Below are all the sections and options you can have in ~/.streamlit/config.toml.
[global]
# By default, Streamlit checks if the Python watchdog module is available and, if not, prints a warning asking for you to install it. The watchdog module is not required, but highly recommended. It improves Streamlit's ability to detect changes to files in your filesystem.
# If you'd like to turn off this warning, set this to True.
# Default: false
disableWatchdogWarning = false
# If True, will show a warning when you run a Streamlit-enabled script via "python my_script.py".
# Default: true
showWarningOnDirectExecution = true
[logger]
# Level of logging: 'error', 'warning', 'info', or 'debug'.
# Default: 'info'
level = "info"
# String format for logging messages. If logger.datetimeFormat is set, logger messages will default to `%(asctime)s.%(msecs)03d %(message)s`. See [Python's documentation](https://docs.python.org/2.6/library/logging.html#formatter-objects) for available attributes.
# Default: None
messageFormat = "%(asctime)s %(message)s"
[client]
# Whether to enable st.cache.
# Default: true
caching = true
# If false, makes your Streamlit script not draw to a Streamlit app.
# Default: true
displayEnabled = true
# Controls whether uncaught app exceptions are displayed in the browser. By default, this is set to True and Streamlit displays app exceptions and associated tracebacks in the browser.
# If set to False, an exception will result in a generic message being shown in the browser, and exceptions and tracebacks will be printed to the console only.
# Default: true
showErrorDetails = true
[runner]
# Allows you to type a variable or string by itself in a single line of Python code to write it to the app.
# Default: true
magicEnabled = true
# Install a Python tracer to allow you to stop or pause your script at any point and introspect it. As a side-effect, this slows down your script's execution.
# Default: false
installTracer = false
# Sets the MPLBACKEND environment variable to Agg inside Streamlit to prevent Python crashing.
# Default: true
fixMatplotlib = true
# Run the Python Garbage Collector after each script execution. This can help avoid excess memory use in Streamlit apps, but could introduce delay in rerunning the app script for high-memory-use applications.
# Default: true
postScriptGC = true
[server]
# List of folders that should not be watched for changes. This impacts both "Run on Save" and @st.cache.
# Relative paths will be taken as relative to the current working directory.
# Example: ['/home/user1/env', 'relative/path/to/folder']
# Default: []
folderWatchBlacklist = []
# Change the type of file watcher used by Streamlit, or turn it off completely.
# Allowed values: * "auto" : Streamlit will attempt to use the watchdog module, and falls back to polling if watchdog is not available. * "watchdog" : Force Streamlit to use the watchdog module. * "poll" : Force Streamlit to always use polling. * "none" : Streamlit will not watch files.
# Default: "auto"
fileWatcherType = "auto"
# Symmetric key used to produce signed cookies. If deploying on multiple replicas, this should be set to the same value across all replicas to ensure they all share the same secret.
# Default: randomly generated secret key.
cookieSecret = "c79e251310888b7b7e1ed34d0e3cc2c636ca779eff64b4effac90f27e329d7af"
# If false, will attempt to open a browser window on start.
# Default: false unless (1) we are on a Linux box where DISPLAY is unset, or (2) server.liveSave is set.
headless = false
# Automatically rerun script when the file is modified on disk.
# Default: false
runOnSave = false
# The address where the server will listen for client and browser connections. Use this if you want to bind the server to a specific address. If set, the server will only be accessible from this address, and not from any aliases (like localhost).
# Default: (unset)
#address =
# The port where the server will listen for browser connections.
# Default: 8501
port = 8501
# The base path for the URL where Streamlit should be served from.
# Default: ""
baseUrlPath = ""
# Enables support for Cross-Origin Request Sharing (CORS) protection, for added security.
# Due to conflicts between CORS and XSRF, if `server.enableXsrfProtection` is on and `server.enableCORS` is off at the same time, we will prioritize `server.enableXsrfProtection`.
# Default: true
enableCORS = true
# Enables support for Cross-Site Request Forgery (XSRF) protection, for added security.
# Due to conflicts between CORS and XSRF, if `server.enableXsrfProtection` is on and `server.enableCORS` is off at the same time, we will prioritize `server.enableXsrfProtection`.
# Default: true
enableXsrfProtection = true
# Max size, in megabytes, for files uploaded with the file_uploader.
# Default: 200
maxUploadSize = 200
# Enables support for websocket compression.
# Default: true
enableWebsocketCompression = true
[browser]
# Internet address where users should point their browsers in order to connect to the app. Can be IP address or DNS name and path.
# This is used to: - Set the correct URL for CORS and XSRF protection purposes. - Show the URL on the terminal - Open the browser - Tell the browser where to connect to the server when in liveSave mode.
# Default: 'localhost'
serverAddress = "localhost"
# Whether to send usage statistics to Streamlit.
# Default: true
gatherUsageStats = true
# Port where users should point their browsers in order to connect to the app.
# This is used to: - Set the correct URL for CORS and XSRF protection purposes. - Show the URL on the terminal - Open the browser - Tell the browser where to connect to the server when in liveSave mode.
# Default: whatever value is set in server.port.
serverPort = 8501
[mapbox]
# Configure Streamlit to use a custom Mapbox token for elements like st.pydeck_chart and st.map. To get a token for yourself, create an account at https://mapbox.com. It's free (for moderate usage levels)!
# Default: ""
token = ""
[deprecation]
# Set to false to disable the deprecation warning for the file uploader encoding.
# Default: true
showfileUploaderEncoding = true
# Set to false to disable the deprecation warning for using the global pyplot instance.
# Default: true
showPyplotGlobalUse = true
# Name of the AWS S3 bucket to save apps.
# Default: (unset)
#bucket =
# URL root for external view of Streamlit apps.
# Default: (unset)
#url =
# Access key to write to the S3 bucket.
# Leave unset if you want to use an AWS profile.
# Default: (unset)
#accessKeyId =
# Secret access key to write to the S3 bucket.
# Leave unset if you want to use an AWS profile.
# Default: (unset)
#secretAccessKey =
# The "subdirectory" within the S3 bucket where to save apps.
# S3 calls paths "keys" which is why the keyPrefix is like a subdirectory. Use "" to mean the root directory.
# Default: ""
keyPrefix = ""
# AWS region where the bucket is located, e.g. "us-west-2".
# Default: (unset)
#region =
# AWS credentials profile to use.
# Leave unset to use your default profile.
# Default: (unset)
#profile =
[theme]
# The preset Streamlit theme that your custom theme inherits from. One of "light" or "dark".
#base =
# Primary accent color for interactive elements.
#primaryColor =
# Background color for the main content area.
#backgroundColor =
# Background color used for the sidebar and most interactive widgets.
#secondaryBackgroundColor =
# Color used for almost all text.
#textColor =
# Font family for all text in the app, except code blocks. One of "sans serif", "serif", or "monospace".
#font =
I don’t get any errors when I use that TOML file. Can you please post the version of the config.toml
that produces the error?
Thank you!
I did not understand what you mean by the version of config.toml
.
Each time i try to create the file by typing this statement streamlit config show > ~/.streamlit/config.toml
than try to run the app the system crash and display the error that i showed above.
Based on this post everyone has delete the config file and everything work again.
How to fix this because i am stuck in the old version of streamlit. v 0.84
Thanks for stopping by! We use cookies to help us understand how you interact with our website.
By clicking “Accept all”, you consent to our use of cookies. For more information, please see our privacy policy.
Strictly necessary cookies
These cookies are necessary for the website to function and cannot be switched off. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms.
Performance cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us understand how visitors move around the site and which pages are most frequently visited.
Functional cookies
These cookies are used to record your choices and settings, maintain your preferences over time and recognize you when you return to our website. These cookies help us to personalize our content for you and remember your preferences.
Targeting cookies
These cookies may be deployed to our site by our advertising partners to build a profile of your interest and provide you with content that is relevant to you, including showing you relevant ads on other websites.
Reject all
Accept all