TY - GEN
T1 - Distributed stream processing analysis in high availability context
AU - Gorawski, Marcin
AU - Marks, Pawel
PY - 2007
Y1 - 2007
N2 - Not so long ago data warehouses were used to process data sets loaded periodically during ETL process (Extraction, Transformation and Loading). We could distinguish two kinds of ETL processes: full and incremental. Now we often have to process real-time data and analyse them almost on-the-fly, so the analyses are always up to date. There are many possible applications for real-time data warehouses. In most cases two features are important: delivering data to the warehouse as quick as possible, and not losing any tuple in case of failures. In this paper we propose an architecture for gathering and processing data from geographically distributed data sources. We present theoretical analysis, mathematical model of a data source, some rules of system modules configuration and results of experiments. At the end of the paper our future plans are described briefly.
AB - Not so long ago data warehouses were used to process data sets loaded periodically during ETL process (Extraction, Transformation and Loading). We could distinguish two kinds of ETL processes: full and incremental. Now we often have to process real-time data and analyse them almost on-the-fly, so the analyses are always up to date. There are many possible applications for real-time data warehouses. In most cases two features are important: delivering data to the warehouse as quick as possible, and not losing any tuple in case of failures. In this paper we propose an architecture for gathering and processing data from geographically distributed data sources. We present theoretical analysis, mathematical model of a data source, some rules of system modules configuration and results of experiments. At the end of the paper our future plans are described briefly.
UR - https://www.scopus.com/pages/publications/34548170287
U2 - 10.1109/ARES.2007.72
DO - 10.1109/ARES.2007.72
M3 - Conference contribution
AN - SCOPUS:34548170287
SN - 0769527752
SN - 9780769527758
T3 - Proceedings - Second International Conference on Availability, Reliability and Security, ARES 2007
SP - 61
EP - 68
BT - Proceedings - The Second International Conference on Availability, Reliability and Security, ARES 2007
PB - IEEE Computer Society
T2 - 2nd International Conference on Availability, Reliability and Security, ARES 2007
Y2 - 10 April 2007 through 13 April 2007
ER -