Quantifying volume, velocity, and variety to support (Big) data-intensive application development

Abstract

© 2017 IEEE. In the era of digital economies, data can be considered as the new commodity, fueling the next-generation software services and applications. Increasing amounts of data, generated on a daily basis by various domains, such as social networks, stock exchanges, the Internet of Things, and cyber-physical systems, are soon expected to exceed the yottabyte1 frontier. To process this overwhelming amount, Big Data solutions are being developed to enable a new generation of data-centric/data-intensive applications (DIAs) and services. However, many of such applications currently fail to meet the increasingly demanding data management requirements. In particular, proper techniques and tools to support architects and developers in DIA design are required to cope with these pressing Big Data challenges. This paper makes an initial step in this direction, aiming at reducing the gap between the architects and DIAs they have to develop. The proposed approach extends the conventional Big Data process workflow with a way of capturing and modeling the 'three Vs' of Big Data (i.e. volume, velocity, and variety) to provide useful insights on the overall process, knowing the behavior of its individual components. Starting from the V-attributes of the Big Data process components, the proposed framework provides an estimation of its V-metrics by evaluating a performance model generated from the process. To demonstrate the feasibility and the effectiveness of the approach, a case study on a computer vision DIA is reported.