I'll share I'm having a variant of the same issue. I have a varchar field in the form YYYYMMDD which I'm trying to join to another varchar field from another table in the form of MM/DD/YYYY. Does anyone know of a way to do this in SPARK SQL without splitting the YEAR, MONTH and Day and concatenating them together? I've tried date_format, to_date, date, cast, and to_timestamp functions or different combinations of them, but haven't yet succeeded. Any help is appreciated. Thanks!

If there isn’t a group near you, start one and help create a community that brings people together. Request a New Group Paramter Type while Executing Rest API of Execute a SQL statement POST /api/2.0/sql/statements/ in Data Engineering 08-26-2024 Converting databricks query output in JSON and creating end point in Data Engineering 08-22-2024 Cannot use Delta Table columns containing struct with date fields in Power BI in Data Engineering 08-19-2024 © Databricks 2024. All rights reserved. Apache, Apache Spark, Spark and the Spark logo are trademarks of the Apache Software Foundation.

Privacy Notice

| Terms of Use

| Your Privacy Choices

| Your California Privacy Rights