Using the Capabilities of XML and Materialized Views in Creating a Near Real-Time Data Warehouse
Subject Areas : electrical and computer engineeringS. M. Shafaei 1 , S. M. Shafaei 2
1 -
2 -
Keywords: Near real-time data warehouse, materialized view, XML, XSTL, XQuery,
Abstract :
A major challenge in the field of platforms and applications is how to display and combine the results of real-time and static partitions, as well as reduce the response time of on-line analytical processing queries in a near real-time data warehouse. So appropriate content in a near real-time data warehouse can be produced through a common interface for the results of queries. This article provides an architecture that includes an XML/XSLT interface approach to generate appropriate content and also creating materialized views in the client side. In this architecture, providing a model-based HTML output, distribution and composition of results, are presented. In addition, two parallel approaches for incorporating the results of real-time and static partitions of near real-time data warehouse architecture are proposed. In the proposed architecture, the fundamental role of XML and its related technologies, the production and maintenance of content in near real-time data warehouse is determined. The results show that response time of on-line analytical processing queries via materialized views in the server and client side is reduced. Introduced functions for selecting materialized views in the both of client and server sides improve the storage space.
[1] Y. Zhu, Lei An, and S. Liu, "Data updating and query in real-time data warehouse system," in Proc. of Int. Conf. on Computer Science and Software Engineering, vol. 5, pp. 1295-1297, Dec. 2008.
[2] R. J. Santos and J. Bernardino, "Real-time data warehouse loading methodology," in Proc. of ACM Int. Symp. on Database Engineering & Applications, pp. 49-58, Sept. 2008.
[3] A. Cuzzocrea, N. Ferreira, and P. Furtado, "Enhancing traditional data warehousing architectures with real-time capabilities," in Proc. of Int. Symp. on Methodologies for Intelligent Systems, pp. 456-465, Jun. 2014.
[4] M. Obali, B. Dursun, Z. Erdem, and A. K. Görür, "A real time data warehouse approach for data processing," in Proc. of Signal Processing and Communications Applications Conf., SIU'13, 4 pp., Apr. 2013.
[5] J. Zuters, "Near real-time data warehousing with multi-stage trickle and flip," in Proc. of Int. Conf. on Business Informatics Research, . pp. 73-82, Oct. 2011.
[6] T. Winsemann, V. Koppen, and G. Saake, "A layered architecture for enterprise data warehouse systems," in Proc. of Int.Conf. on Advanced Information Systems Engineering, pp. 192-199, Jun. 2012.
[7] Y. Sharma, R. Nasri, and K. Askand, "Building a data warehousing infrastructure based on service oriented architecture," in Proc. of IEEE Int. Conf. on Cloud Computing Technologies, Applications and Management, ICCCTAM'12, pp. 82-87, Dec. 2012.
[8] V. Gonzalez-Castro, L. M. MacKinnon, and M. del Pilar Angeles, "An alternative data warehouse reference architectural configuration," in Proc. of British National Conf. on Databases, pp. 33-41, Jul. 2009.
[9] Y. Mao, et al., "Dynamic mirror based real-time query contention solution for support big real-time data analysis," in Proc. of 2nd Int. Conf. on Information Technology and Electronic Commerce, ICITEC'14, pp. 229-233, Dec. 2014.
[10] W. Qu, V. Basavaraj, S., Shankar, and S. Dessloch, "Real-time snapshot maintenance with incremental ETL pipelines in data warehouses," in Proc. of Int. Conf. on Big Data Analytics and Knowledge Discovery, pp. 217-228, Sept. 2015.
[11] Z. Lin, Y. Lai, C. Lin, Y. Xie, and Zou Q, "Maintaining internal consistency of report for real-time OLAP with layer-based view," in Proc. of Asia-Pacific Web Conf., pp. 143-154, Apr. 2011.
[12] I. Hamdi, E. Bouazizi, and J. Feki, "Dynamic management of materialized views in real-time data warehouses," in Proc. of 6th Int. Conf. on Soft Computing and Pattern Recognition, SoCPaR'14, pp. 168-173, Aug. 2014.
[13] T. Jain, "Refreshing data warehouse in near real-time," Int. J. of Computer Applications, vol. 46, no. 18, pp. 24-28, May 2012.
[14] M. Asif Naeem, G. Dobbie, and G. Webber, "An event-based near real-time data integration architecture," in Proc. of 12th Enterprise Distributed Object Computing Conf. Workshops, pp. 401-404, Sept. 2008.
[15] S. Yi Chuan and X. Yao, "Research of real-time data warehouse storage strategy based on multi-level caches," Physics Procedia, vol. 25, pp. 2315-2321, Jan. 2012.
[16] R. J. Santos, J. Bernardino, and M. Vieira, "24/7 real-time data warehousing: a tool for continuous actionable knowledge," in Proc. of 35th Annual Computer Software and Applications Conf., pp. 279-288, Jul. 2011.
[17] N. Ferreira, P. Martins, and P. Furtado, "Near real-time with traditional data warehouse architectures: factors and how-to," in Proc. of ACM 17th Int. Database Engineering & Applications Symp., pp. 68-75, Oct. 2013.
[18] P. O'Neil, E. O'Neil, X. Chen, and S.Revilak, "The star schema benchmark and augmented fact table indexing," in Proc. of Technology Conf. on Performance Evaluation and Benchmarking, pp. 237-252, Aug. 2009.
[19] M. Nguyen Tho and A. Min Tjoa, "Zero-latency data warehousing for heterogeneous data sources and continuous data streams," in Proc. of 5th Int. Conf. on Information Integration and Web-Based Applications Services, pp. 55-64, Sept. 2003.
[20] L. Golab and T. Johnson, "Data stream warehousing," in Proc. of IEEE 30th Int. Conf. on Data Engineering, pp. 949-952, Mar./Apr. 2014.
[21] M. Gorawski and A. Gorawska, "Research on the stream ETL process," Proc. of Int. Conf. Beyond Databases, Architectures and Structures, pp. 61-71, Jan. 2014.
[22] R. Abrahiem, "A new generation of middleware solutions for a near-real-time data warehousing architecture," in Proc. of IEEE Int. Conf. on Electro/Information Technologypp. 192-197, May 2007.
[23] M. K. Sohrabi and V. Ghods, "Materialized view selection for data warehouse using frequent itemset mining," J. of Computers, vol. 11, no. 1, pp. 140-148, Mar. 2016.
[24] A. Gosain, "Materialized cube selection using particle swarm optimization algorithm," Procedia Computer Science, vol. 79, pp. 2-7, Jan. 2016.
[25] T. Win, "Conversion of XML schema to data warehouse schema using automatic approach," International J. of Computer Applications, vol. 108, no. 15, pp. 12-18, Dec. 2014.
[26] Z. Ouaret, R. Chalal, and O. Boussaid, "An approach for generating an XML data warehouse schema using model transformation language," Journal of Digital Information Management, vol. 12, no. 6, pp. 407-420, Dec. 2014.