MDACA Big Data Virtualization (BDV) is an enterprise grade MPP (massively parallel processing) federated SQL query engine supporting the connected digital infrastructure.
BDV provides a single view of enterprise data while concealing the technical complexities of database types, data locations, and data transformations regardless of where the data resides.
BDV supports data queries across systems without data copy and replication, thereby bolstering master data management, Analytics, Insights, AI/ML, and legacy data migration initiatives while reducing cost. It is operationally optimized for the cloud and offers a comprehensive data abstraction, federation, integration, and transformation layer. As an example, BDV easily integrates with AWS core service offerings such as Redshift, Aurora, AWS RDS, AWS Lake Formation, Glue, S3, Amazon Quick Sights, EMR, and SageMaker.
Additionally, it easily integrates with MDACA enterprise tools through single sign-on (sso) with MDACA products such as Data Explorer, Cloud Storage Explorer, Data Lake, and Synthetic Data Engine – all designed to provide advanced enterprise capabilities for big data solutions.
Enterprise Grade, Leveraging Industry Standards
BDV provides organizations with a logical data layer that integrates enterprise data across disparate systems and manages the unified data for centralized access.
Based on the industry leading Trino.io distributed SQL query engine, MDACA BDV has been upgraded to support enterprises with advanced DoD-grade security features, additional connectors, functions, and performance optimizations. MDACA BDV also provides KeyCloak.org (supporting enterprise single-sign-on and providing user federation, strong authentication, user management, fine-grained authorization, and more) and Ranger.org (fine grained authorization to perform a specific action) as part of the baseline AMI configuration, supporting security and master data management needs.
Data Connectors
BDV data connectors are similar to database drivers, but designed to connect to support both structured and unstructured data. Unstructured data can include file types such as csv, parquet avro, etc.. Additionally BDV is fully extendable to allow the development of custom connectors meeting a wide range of advanced business needs. This allows building special adaptors for file systems and object stores, NoSQL stores, relational database systems, and custom services as building specific adapters for secure data at rest such as accessing quantum encrypted data at rest. A custom connector can be created for any data that can be mapped into relational concepts such as tables, columns, and rows. BDV comes with a number of built-in connectors for a variety of industry leading databases and query engines. Secure connectors include, but are not limited to, the following:
- MDACA Data Lake
- Amazon Redshift
- Amazon Aurora
- SQL Server
- Oracle
- Percona
- Cassandra
- MySQL
- Postgres
- Hive
- MariaDB
- Amazon OpenSearch
- AWS Lake Formation
- Amazon S3
- Snowflake
- BigQuery
- SQLite
- Kafka
- Kineses
- Amazon RDS
- Any ODBC Database
Key Features
Designed to provide a single view of enterprise data while hiding the technical complexities of database types, data locations, and data transformations making it easy for business owners to understand.
Logical Data Layer
Provides a virtual approach to accessing,
managing, and delivering data.
Legacy Data Migration Support
Reduce risk of system modernization by updating
business applications while replacing legacy
systems.
Query Federation
Access data from multiple systems within a
single query.
Eliminate Data Silos
Delivers integrated information while reducing data
silos, allowing data to remain in source systems
and reducing number of data copies.
Data Management Support
Provides a centralized secure layer to catalog,
search, discover, and govern unified data and its
relationships.
Advanced Query Support
Support American National Standards Institute (ANSI) Structured Query Language (SQL) semantics, including complex queries, aggregations, and sub-queries.
Data Integration
Integrates data across the enterprise systems
supporting a wide range of data formats and
sources.
Scaled to Support Business and Security Needs
Scale easily to run large queries and on-demand
clusters coupled with fine grained security and
privacy controls.
High Performance Queries
Supports highly parallel and distributed queries
built from the ground up for efficient, low latency
analytics.
BDV Integration with Data Explorer
MDACA Data Explorer (DE) is a lightweight, web-based universal SQL client that allows you to query and browse any Open Database Connectivity (ODBC) compliant databases. It also provides a data connector designed for integration with BDV, expanding the power to query your data across the enterprise regardless where the data resides. All that is needed is for that data source to be connected to DBV..
Data Explorer combined with BDV enables:
- Leveraging an easy to use web-based user interface to support executing high performance data queries across multiple environments with robust Massively Parallel Processing (MPP)
- Support for Role Based Access Control (RBAC) to your data, including row and column level access security
- Easily accessible through standard Structured Query Language (SQL) and/or Web Services interfaces for virtual data utilization and sharing
- Advanced query capabilities across the enterprise
BDV Integration with Data Storage Explorer
MDACA Cloud Storage Explorer (CSE) is a web-based file explorer that is designed to allow you to manage files across a wide range of cloud storage providers through a single integrated solution. This includes: AWS S3, Azure Blob Storage, Google GCP Cloud Storage, and SharePoint Document Libraries as well as Windows and Linux shares.
Data Storage Explorer combined with BDV enables:
- Provides the tools to support enterprise data scientists, developers, testers, and business analysts in working securely with data files located across the enterprise in datalakes and files systems.
- CSE provides one click functionality to add unstructured data (file types as csv, parquet avro, etc..) to BDV, enabling applications to query the data from the files without the need to load the data to a traditional database allowing query-based applications and tools to quickly access the data.
BDV De-Identified Data and Integration with Synthetic Data Engine
BDV out-of-the-box functionality provides the ability to specify user groups and user communities that have access only to de-identified data when querying the data. This is accomplished by adding business rules (e.g., data masking) in real-time to the data being queried while applying data access and visibility rules down to the column level. When more business and master data management policies have requirements for synthetic data, the BDV integration with MDACA Synthetic Data Engine provides advanced capabilities in supporting operations such as creating a development data environment, enhancing application testing and supporting insights testing with AI/ML development.
The Synthetic Data Engine (SDE) is a web-based synthetic data generator designed to seamlessly create meaningful, intelligent synthetic data without compromising the privacy of corresponding sensitive, real data.The MDACA SDE leverages intelligent AI-driven synthetic data generation to create non-reversible, artificially generated data with the statistical characteristics and correlations of the real data.
BDV Integration with Policy Manager
The MDACA BDV Policy Manager is designed to provide advanced granular security access to the data being accessed/managed through the BDV platform. It fully integrates with industry leading identity management systems to provide granular security to the column and row level on a given data connector.
The Synthetic Data Engine (SDE) is a web-based synthetic data generator designed to seamlessly create meaningful, intelligent synthetic data without compromising the privacy of corresponding sensitive, real data.The MDACA SDE leverages intelligent AI-driven synthetic data generation to create non-reversible, artificially generated data with the statistical characteristics and correlations of the real data.
BDV Integration with KeyCloak
MDACA BDV is designed to work with KeyCloak and other identity management providers to provide user federation, strong authentication, user management, fine-grained authorization, and more.
BDV Quick Start Program
We understand organization needs will differ based on business needs, budgets, interconnected partners, and ramp up timelines and requirements. The BDV Quick Start Program is designed in collaboration with our industry partners to provide the agile approach to support the quick secure establishment of the baseline approach to meet your data access needs and support the onboarding and training of your teams.
Training and Support
Our training and support program can be tailored to meet your needs. In addition to our staff that specialize in big data solutions, we have direct access to our industry partnerships that provide us access to a wide pool of resources that can tailor training specifically to your business and team’s needs. We offer a number of options in support of your training needs.