Archive Test Pre-requisites

Introduction

Collection page for pre-requisites necessary to be able to carry out more advanced tests on the ALMA archive. The items listed here are concerning a number of other subsystems, since the archive will need to have a lot of realistic data in all stores in order to be able to derive realistic numbers on performance, throughpout and query capabilities.

The test cases mentioned here are only the ones, which require either interaction or input with or from other subsystems. It contains pure archive test cases, but also test cases, which go beyond the archive and ar more system test cases, involving several subsystems or testing the efficiency of the deployment of the subsystems. The page does not give all the details of the test cases, since that will go to a separate document. The prerequisites are subject to change according to the detailed design of the test cases. Since the archive is not responsible for the design of the system test cases the prerequistes listed in those test cases are just indicative.

Test Cases

Bulk data stream

---

Bulk data multiplexing

Name	Bulk data multiplexing
Category	System Test
Subject	A/V streaming
Subsystems concerned	ACS, Correlator, Archive, TelCal, QuickLook
Test design	HLA/Archive
Test execution	ITS/HLA
Description	The main concern of the BulkStore is the required throughput between the Correlator and the final location of the data on NGAS. The Archive subsystem does carry out tests on the individual components of the archive part of this chain and there are also tests of the plain A/V streaming. More advanced testing is necessary at some point to simulate the required multiplexing of the data from the Correlator to the Archive, TelCal and QuickLook subsystems. This involves four subsystems and the archive is the least critical one in terms of timing requirements.
Goal	Verify and quantify the ability of the adopted A/V streaming solution to deliver the required data rate simultanously to Archive, TelCal, QuickLook.

Number	Prerequisite	Subsystems involved
1	Stable and agreed A/V streaming library	ACS, Correlator, Archive
2	Sender and receiver implementations for all subsystems	Correlator, Archive, TelCal, QuickLook
3	Deployment design	HLA, ITS
4	Adequate hardware setup	ITS
5	Definition of the required throughput for the setup in (3)	HLA

Main Page

-- AndreasWicenec - 27 Apr 2005

---

---

Bulk Data Load Balancing

Name	Bulk data Load Balancing
Category	System Test
Subject	A/V streaming, NGAS storage
Subsystems concerned	Archive, Correlator
Test design	Archive
Test execution	Archive
Description	This test
Goal	Verify and quantify the ability NGAS using the adopted A/V streaming solution to store data with the expected maximum throughput for a given deployment and scalability to 60 MB/s and with the required stability (min. 24 hours)

Number	Prerequisite	Subsystems involved
1	Implementation of A/V streaming chain with BulkSender BulkDistributor and BulkReceiver	ACS, Correlator, Archive
2	Implementation of load balancing BulkReceiver	Archive
3	Deployment design	Archive, ACS, ITS
4	Implementation of deployment on adequate hardware setup	ITS, Archive
5	Definition of the required throughput for the given setup in (3)	HLA

Detailed discussion

Archive Tests Main Page

-- AndreasWicenec - 25 Jan 2007

---

MonitorStore

---

Monitor Ingest Rate

Name	Monitor Ingest Rate
Category	Archive Test
Subject	A/V streaming, Transactions
Subsystems concerned	Control, ACS, Archive
Test design	Archive
Test execution	Archive
Description
Goal	Verify and quantify the ability of the MonitorStore to handle the required data rate.

Number	Prerequisite	Subsystems Involved
1	Stable and agreed A/V streaming library	ACS
2	Stable and agreed monitor format, including definition of key structure for monitor devices.	Control
3	Detailed data rate numbers, including expected number of devices and monitor points, data types and formats.	Control
4	Definition of the required rate using (3)	Archive, Control

Main Page

-- AndreasWicenec - 27 Apr 2005

---

Test Monitor Query

Name	Test Monitor Query
Category	Archive Test
Subject	Query usability and performance
Subsystems concerned	Control, Archive
Test design	Archive
Test execution	Archive, External
Description
Goal	Verify and quantify the usability and performance of typical queries against the MonitorStore.

Number	Prerequisite	Subsystems Involved
1	Stable and agreed monitor format, including definition of key structure for monitor devices.	Control
2	Definition of typical queries, including typical time ranges and transaction rates.	Control
3	Definition of the required rate using (2)	Archive, Control

Main Page

-- AndreasWicenec - 27 Apr 2005

---

XMLStore

---

Test APDM Ingest

Name	APDM Ingest
Category	Archive Test
Subject	APDM ingest performance
Subsystems concerned	OT, Archive, SSR
Test design	OT, Archive
Test execution	OT, Archive, External
Description	Test covers the data structures as defined by ObsPrep. Since this is a hierarchical, recursive structure the actual performance is best being verified using realistic data. In preparation of the APDM query test, it would be benefitial to get complete real-world projects with all parameters, SBs and scripts included.
Goal	Verify and quantify the ingest performance of APDM data structures.
Notes	This test can be done in a staged approach, first using dummy incomplete APDM, and then upgrading to real projects, like the DSRPs.

Number	Prerequisite	Subsystems Involved
1	Stable and agreed APDM format.	ObsPrep
2	Definition of the required rate using the DSRP numbers (?)	Offline, SSR

Main Page

-- AndreasWicenec - 27 Apr 2005

---

Test APDM Query

Name	APDM Query
Category	System Test
Subject	APDM query usability and performance
Subsystems concerned	Archive, and all others, depending on complexity.
Test design	HLA
Test execution	ITS/HLA
Description	Test covers the retrieval of the data structures as defined by ObsPrep. Since this is a hierarchical, recursive structure the actual performance is best being verified using realistic data. The level of complexity can be increased step-wise, i.e. in the first stage just (dummy) OT APDM, second stage with referenced (dummy) exec records (Control) and third stage with referenced ASDM and bulk entities.
Goal	Verify and quantify the ingest performance of APDM data structures.
Notes	Query optimisation in the archive will only be started once the APDM is relatively stable and complete. Real tests can only be started once there is a reasonable amount of realistic projects available.

Number	Prerequisite	Subsystems Invloved
1	Stable and agreed APDM format.	ObsPrep
2	Definition of the queries based on TheQueryPage and the ASA Use Cases	Archive
3	Availability of various realistic and complete programs	SSR using OT (?)
4	Availability of exec records	Control
5	Availability of ASDM entities	Capture
6	Availability of bulk entities	Correlator

Main Page

-- AndreasWicenec - 27 Apr 2005

---

Test ASDM Ingest

Name	ASDM Ingest
Category	Archive/System Test
Subject	ASDM ingest performance
Subsystems concerned	Control(Capture), Archive
Test design	Archive, Control
Test execution	Archive
Description	Test covers the science data structures as defined by Offline and HLA. Since this is a rather complex structure the actual performance is best being verified using realistic data. In preparation of the ASDM query test, it would be benefitial to get complete real-world projects catured with all parameters included.
Goal	Verify and quantify the ingest performance of ASDM data structures.
Notes	This test can be done in a staged approach, first using dummy incomplete ASDM, and then upgrading to real projects, like the DSRPs.

Number	Prerequisite	Subsystems Involved
1	Stable and agreed ASDM format.	Offline
2	Evaluation of the required rate using the DSRPs (?)	SSR

Main Page

-- AndreasWicenec - 27 Apr 2005

---

Test ASDM Query

Name	ASDM Query
Category	System Test
Subject	ASDM query usability and performance
Subsystems concerned	Archive, and all others, depending on complexity.
Test design	HLA
Test execution	ITS/HLA
Description	Test covers the retrieval of the data structures as defined by ObsPrep. Since this is a hierarchical, recursive structure the actual performance is best being verified using realistic data. The level of complexity can be increased step-wise, i.e. in the first stage just (dummy) Capture ASDM, second stage with reference (dummy) bulk entities and third stage with simulated or real bulk entities.
Goal	Verify and quantify the query functionality against ASDM data structures.
Notes	Query optimisation in the archive will only be started once the ASDM is relatively stable and complete. Real tests can only be started once there is a reasonable amount of realistic projects available.

Number	Prerequisite	Subsystems Invloved
1	Stable and agreed ASDM format.	Offline
2	Definition of the queries based on TheQueryPage and the ASA Use Cases	Archive
3	Availability of various realistic and complete captured program	HLA
4	Availability of exec records	Control
5	Availability of APDM entities	ObsPrep, SSR (DSRPSs))
6	Availability of bulk entities	Correlator

Main Page

-- AndreasWicenec - 27 Apr 2005

Science Archive

---

Test ASA Query

Name	ASA Query
Category	Archive Test
Subject	ASA query usability and performance
Subsystems concerned	Archive, SSR
Test design	Archive
Test execution	Archive, SSR, External
Description	Test covers the retrieval of the data structures as defined by ASA. Since this is complex structure the actual performance and usability is best being verified using realistic data. The level of complexity can be increased according to the priorities defined in the ASA use-cases.
Goal	Verify and quantify the query usability and performance of ASA data structures.
Notes	Query optimisation in the archive will only be started once the ASA is relatively stable and complete. Real tests can only be started once there is a reasonable amount of complete realistic projects available. The usability could be evaluated before data is actually available.

Number	Prerequisite	Subsystems Invloved
1	Stable and agreed ASA data model.	Archive
2	Definition of the queries based on the ASA Use Cases	Archive
3	Availability of various realistic and complete programs	SSR using OT?
4	Availability of exec records	Control
5	Availability of ASDM entities	Capture
6	Availability of bulk entities	Correlator

Main Page

-- AndreasWicenec - 27 Apr 2005

---

Archive Concurrency

---

Test Archive Concurrency

Name	Archive Concurrency
Category	System Test
Subject	Archive stress test with concurrent access.
Subsystems concerned	Archive, HLA, ITS
Test design	HLA
Test execution	ITS/HLA
Description	Test covers the ingest and retrieval of any kind of data in a concurrent way. Concurrent performance is expensive, because it needs first good planning and then optimal deployment of the different parts and clients.
Goal	Verify and quantify the archive performance under stress.
Notes	Concurrent access to the archive has to be planned, because this will cause the highest impact on the raw performance figures from all the other individual tests.

Number	Prerequisite	Subsystems Invloved
1	Analysis of the ALMA data flow model, including rate and timing analysis.	HLA
2	Availability of either real or simulated clients, which implement the model from (1)	ITS
3	Realistic hard- and software deployment	ITS

Main Page

-- AndreasWicenec - 27 Apr 2005

---

-- AndreasWicenec - 27 Apr 2005