With the ongoing digital transformation, there are large amounts of data being generated by individuals, IoT, and organizations. Due to security and privacy concerns, currently these data are siloed, even though there is a strong motivation to utilize data across organizations to obtain valuable data-driven insights. To enable such data utilization, we have developed a new system for secure data exchange between organizations in a consortium with access control using Blockchain. In this system, the data providers can register metadata along with access control policy associated with the actual data. This metadata and access control is recorded on the distributed ledger, while the actual data remains off-chain and stored locally. The data consumers can discover the different data available in the consortium based on the metadata. When a consumer requests for data, the system uses smart contract to verify against the data’s access control policy, and if allowed, seamlessly establishes secure overlay connection on the Internet to exchange data from the provider. We will present the system architecture implemented using Hyperledger Fabric, demonstrate its features and detail a recent use case of a consortium of multiple organizations, such as real estate, mobile operator, in Tokyo area to securely share data based on access control policy and gain insights for city planning. Also, to overcome limitations of the node scalability in Fabric, we have implemented a multi-cluster exchange system with metadata shared across clusters. Lastly, we will show how to extend this system to realize privacy-preserving decentralized machine learning by training local AI models with distributed on-premises data at each organization and combining these models to achieve better performance.
