pyspark.sql.functions.to_timestamp

pyspark.sql.functions.to_timestamp(col: ColumnOrName, format: Optional[str] = None) → pyspark.sql.column.Column

Converts a Column into pyspark.sql.types.TimestampType using the optionally specified format. Specify formats according to datetime pattern. By default, it follows casting rules to pyspark.sql.types.TimestampType if the format is omitted. Equivalent to col.cast("timestamp").

New in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str

column values to convert.

format: str, optional

format to use to convert timestamp values.

Returns

Column

timestamp value as pyspark.sql.types.TimestampType type.

Examples

>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(to_timestamp(df.t).alias('dt')).collect()
[Row(dt=datetime.datetime(1997, 2, 28, 10, 30))]

>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(to_timestamp(df.t, 'yyyy-MM-dd HH:mm:ss').alias('dt')).collect()
[Row(dt=datetime.datetime(1997, 2, 28, 10, 30))]

previous

pyspark.sql.functions.to_unix_timestamp

next

pyspark.sql.functions.to_timestamp_ltz