spark dataframe 每一列的空字符(或其它值)替换成null

var offlineData: DataFrame = ......

val offlineDataFieldName: Array[String] = offlineData.schema.names
    (0 until offlineDataFieldName.length).foreach(idx => {
      offlineData = offlineData.withColumn(
        offlineDataFieldName.apply(idx)
        , when(col(offlineDataFieldName.apply(idx)) === "" or col(offlineDataFieldName.apply(idx)) === "其它值", lit(null)).otherwise(col(offlineDataFieldName.apply(idx)))
      )
    })

参考资料：
https://www.thinbug.com/q/44102133

这篇好文章是转载于：学新通技术网