Sharing Big Data Safely ― Managing Data Security

Many big data-driven companies today are moving to protect certain types of data against intrusion, leaks, or unauthorized eyes. But how do you lock down data while granting access to people who need to see it? In this practical book, authors Ted Dunning and Ellen Friedman offer two novel and practical solutions that you can implement right away.

Ideal for both technical and non-technical decision makers, group leaders, developers, and data scientists, this book shows you how to:

• Share original data in a controlled way so that different groups within your organization only see part of the whole. You’ll learn how to do this with the new open source SQL query engine Apache Drill.
• Provide synthetic data that emulates the behavior of sensitive data. This approach enables external advisors to work with you on projects involving data that you can't show them.

If you’re intrigued by the synthetic data solution, explore the log-synth program that Ted Dunning developed as open source code (available on GitHub), along with how-to instructions and tips for best practice. You’ll also get a collection of use cases.

Providing lock-down security while safely sharing data is a significant challenge for a growing number of organizations. With this book, you’ll discover new options to share data safely without sacrificing security.

Ted Dunning is Chief Applications Architect at MapR Technologies and active in the open source community.

He currently serves as VP for Incubator at the Apache Foundation, as a champion and mentor for a large number of projects, and as committer and PMC member of the Apache ZooKeeper and Drill projects. He developed the t-digest algorithm used to estimate extreme quantiles. T-digest has been adopted by several open source projects. He also developed the open source log-synth project described in this book.

Ted was the chief architect behind the MusicMatch (now Yahoo Music)and Veoh recommendation systems, built fraud-detection systems forID Analytics (LifeLock), and has issued 24 patents to date. Ted has aPhD in computing science from University of Sheffield. When he’s notdoing data science, he plays guitar and mandolin. Ted is on Twitter as@ted_dunning.

Ellen Friedman is a solutions consultant and well-known speaker and author, currently writing mainly about big data topics. She is a committer for the Apache Drill and Apache Mahout projects. With a PhD in Biochemistry, she has years of experience as a research scientist and has written about a variety of technical topics including molecular biology, nontraditional inheritance, and oceanography. Ellen is also co-author of a book of magic-themed cartoons, A Rabbit Under the Hat. Ellen is on Twitter as @Ellen_Friedman.