How to extend your Data Security at Lakehouse

As you may already know, Lakehouse is an architecture solution developed by Databricks thanks to the Delta Lake file format and built on top of Data Lake. This new approach to data provides many benefits to data teams, but I’m not going to put the spotlight on them at this time, because I’d like to delve into security topic such as encryption of data at rest.

Lakehouse is GDPR compliant, but in contrast to Azure SQL Database there are no native solutions like Dynamic Data Masking or even Always Encrypted. This means that, if you have enough permissions, you will have the opportunity to read personally identifiable information (PII) as plain text via a simple SQL query.

Therefore, the user-defined function (UDF) can make your life easier. You can enable a simple function to apply column-level encryption and reuse it when a sensitive field requires it.

To do this, you just need to import the encryption library and start writing your Python UDF.

Source: Databricks

This Databricks article is worth reading if you want to learn how to put column-level encryption on your Delta Tables to increase the security level of your data.

Publicado por alb3rtoalonso

Soy un enamorado del poder de los datos. Entusiasta de la mejora y formación continua.

Deja una respuesta

Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de

Estás comentando usando tu cuenta de Salir /  Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Salir /  Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Salir /  Cambiar )

Conectando a %s

A %d blogueros les gusta esto: