This page (revision-43) was last changed on 18-Apr-2021 19:03 by JianSun

This page was created on 24-Apr-2008 18:38 by JianSun

Only authorized users are allowed to rename pages.

Only authorized users are allowed to delete pages.

Page revision history

Version Date Modified Size Author Changes ... Change note
43 18-Apr-2021 19:03 5 KB JianSun to previous
42 02-May-2008 17:30 5 KB JianSun to previous | to last
41 01-May-2008 03:17 5 KB David R Williams to previous | to last

Page References

Incoming links Outgoing links
CompressionFactorStudy
CompressionFactorStudy

Version management

Difference between version and

At line 1 changed one line
The compression rate of on board EIS data may vary depending on a few factors such as the observing target (QS, AR, CH, etc), slit/slot selection, exposures, etc.
[{ALLOW edit EISMainUsers}]
[{ALLOW view Anonymous}]
The __compression factor__ of on board EIS data may vary depending on a few factors such as the observing target (QS, AR, CH, etc), slit/slot selection, exposures, etc.
At line 3 changed one line
The purpose of this study is trying to investigate how different compression schemes effect EIS data volume on board and work out a better estimation of compression rate for compression scheme (eg. DPCM, JPEG98, JPEG95, etc.)
The purpose of this study is trying to investigate how different compression schemes effect EIS data volume on board (MDP) and work out a better estimation of compression factor for compression scheme (eg. DPCM, JPEG98, JPEG95, etc.)
At line 5 changed one line
The approach:
!The approach:
At line 7 changed one line
1. to get actual data volume from MDP status information: the inclined curve means data packets from EIS on MDP, the vertical curve means data packets dumped to ground station. So in general, known a raster's start and end time can calculate actual data volume, and then compare it with the designed data volume of this raster to get data compression rate.
1. to get actual data volume from MDP status information: the inclined curve means data packets from EIS on MDP increasing, the vertical curve means data packets dumped to ground station. So in general, knowing a raster's start and end time one can fitstly calculate actual data volume in that duration, and then compare it with the designed data volume in raster's definition, to get data compression factor.
At line 13 changed one line
3. prepare plots based on various factor combinations: compression rate vs. slit/slot, rate vs. target, rate vs. exposures
3. prepare plots based on various factor combinations: compression factor vs. slit/slot, factor vs. target, factor vs. exposures
At line 17 added one line
\\
At line 19 added one line
!!Some preliminary results (plots):
At line 21 added one line
\\
At line 18 removed one line
----
At line 20 changed one line
Some preliminary results (plots):\\
''The investigation here is for date sets obtained mainly between %%(color:#cc0000;)2007-Sep-15 and 2007-Dec-15%%, as EIS operations were most efficient and stable over this duration. The other data sets inluded here are: %%(color:#cc0000;)2008-Mar%% (for JPEG85), %%(color:#cc0000;)2006-Dec%% (for JPEG75), %%(color:#cc0000;)2007-Jan to 2007-Apr%%, and %%(color:#cc0000;)2008-Feb%%.
''
At line 22 changed one line
(The investigation here is for dates between %%(color:#cc0000;)2007-Sep-15 and 2007-Dec-15%%, as EIS is operationally stable over this duration.)
\\
At line 30 added one line
* [DPCM Compression Scheme]
At line 26 removed 5 lines
%%tabbedSection
%%tab-DPCM
%%(text-align:center)
!DPCM compression (all samples)
%%
At line 32 changed one line
[{Image src='images/factor/dpcm/DPCM_only.gif}]
* [JPEG98 Compression Scheme]
At line 34 removed 3 lines
%%(text-align:center)
!DPCM compression vs different target
%%
At line 38 changed one line
DPCM vs. QS
* [JPEG95 Compression Scheme]
At line 40 removed one line
[{Image src='images/factor/dpcm/DPCM_QS_only.gif}]
At line 42 changed one line
DPCM vs. AR
* [JPEG90 Compression Scheme]
At line 44 removed one line
[{Image src='images/factor/dpcm/DPCM_AR_only.gif}]
At line 46 changed one line
DPCM vs. CH
* [JPEG85 Compression Scheme]
At line 48 removed one line
[{Image src='images/factor/dpcm/DPCM_CH_only.gif}]
At line 50 changed one line
DPCM vs. PCH
* [JPEG75 Compression Scheme]
At line 52 changed one line
[{Image src='images/factor/dpcm/DPCM_PCH_only.gif}]
\\
At line 54 changed one line
DPCM vs. FIL
----
The __average values__ of compression factor for each scheme are listed in following %%(color:#c00;)Sortable Table%% (although only a single number can't show the compression factor varation over a large dataset).
At line 56 changed 4 lines
[{Image src='images/factor/dpcm/DPCM_FIL_only.gif}]
%%(text-align:center)
!DPCM compression vs SLA selection
%%sortable
|| Scheme || Total || QS || AR || CH || 1" || 2" || 40" || 266" || 10s || 15s || 20s || 30s || 60S
|DPCM |2.52|2.80|2.80|2.70|3.0|2.96|3.63|2.49|3.08|3.01|2.5|2.6|2.42
|DPCM | | | | | | | |QS|3.14|2.94|2.91|2.52|2.32
|DPCM | | | | | | | |AR|2.96|3.0|2.35|2.5|2.14
|DPCM | | | | | | | |CH| |3.05|2.52|2.88|2.47
|JPEG98|2.56|2.67|2.46|3.44|2.80|2.36|2.56| |3.21|2.54| |2.4|
|JPEG98| | | | | | | |QS|3.32|2.60| |2.43|
|JPEG98| | | | | | | |AR|3.17|2.42| |2.49|
|JPEG98| | | | | | | |CH|3.6| | | |
|JPEG95|6.25|6.12|6.38| |6.23|6.3| | |5.86| |4.60| |
|JPEG95| | | | | | | |AR|5.82| |4.08| |
|JPEG90|8.11|7.39|8.6|8.4|4.85| |5.04|8.34| | | |5.04|
|JPEG90| | | | | | | |QS| | | |3.87|
|JPEG90| | | | | | | |AR| | | |3.81|
|JPEG85|5.56|5.58|4.92|6.21| | |5.56| | | | | |
|JPEG75|11.39| | | | | | |11.39|11.39| | | |
At line 62 changed one line
DPCM vs. 1" slit
However, the number shown above has difference from previous work done by others, eg, Hara'san result:
At line 64 changed one line
[{Image src='images/factor/dpcm/DPCM_1arcsec_only.gif}]
{{{
> 40" SLOT
> DPCM 2.36
> JPEG98 2.70
> JPEG95 3.47
> JPEG92 4.22
> JPEG90 4.63
> JPEG85 5.74
> JPEG75 7.63
> JPEG65 9.43
> JPEG50 12.0
> For 10s exposure time. }}}
At line 66 changed one line
DPCM vs. 2" slit
\\
At line 68 changed one line
[{Image src='images/factor/dpcm/DPCM_2arcsec_only.gif}]
This work is just a start, the method and results shown here need to check again, and compare with other numbers. !As I mentioned above, these are only preliminary results on EIS compression factor. Still there are more need to be done, so any contributions, comments and suggestions are very helpful!\\
At line 70 removed one line
DPCM vs. 40" slit
At line 72 changed one line
[{Image src='images/factor/dpcm/DPCM_1arcsec_only.gif}]
----
At line 74 removed one line
DPCM vs. 266" slit
At line 76 changed one line
[{Image src='images/factor/dpcm/DPCM_266arcsec_only.gif}]
%%information
As for target selection, in this case, I uses value of %%(color:#cc0000;)SCI_OBJ%% keyword instead of %%(color:#cc0000;)TARGET%% keyword as the later is only meaningful at (study/raster) design stage, the actual target is chosen during the EIS planning.
At line 97 added one line
However, sometimes there is no value for SCI_OBJ in EIS fits header, and sometimes there are two many values (see [SCI_OBJ_Example]). Lots of EIS dataset are thrwon away as hardly to decide to which catagroy the SCI_OBJ belongs to, such as QS,AR or CH.
At line 79 removed 2 lines
%%(text-align:center)
!DPCM compression vs SLA selection vs target
At line 82 changed 101 lines
DPCM vs. 1" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_1arcsec_only.gif}]
DPCM vs. 2" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_2arcsec_only.gif}]
DPCM vs. 40" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_40arcsec_only.gif}]
DPCM vs. 266" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_266arcsec_only.gif}]
DPCM vs. 1" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_1arcsec_only.gif}]
DPCM vs. 2" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_2arcsec_only.gif}]
DPCM vs. 40" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_40arcsec_only.gif}]
DPCM vs. 266" slit vs. AR
[{Image src='images/factor/dpcm/DPCM_AR_266arcsec_only.gif}]
DPCM vs. 1" slit vs. QS
[{Image src='images/factor/dpcm/DPCM_QS_1arcsec_only.gif}]
DPCM vs. 2" slit vs. QS
[{Image src='images/factor/dpcm/DPCM_QS_2arcsec_only.gif}]
DPCM vs. 40" slit vs. QS
[{Image src='images/factor/dpcm/DPCM_QS_40arcsec_only.gif}]
DPCM vs. 266" slit vs. QS
[{Image src='images/factor/dpcm/DPCM_QS_266arcsec_only.gif}]
DPCM vs. 1" slit vs. CH
[{Image src='images/factor/dpcm/DPCM_CH_1arcsec_only.gif}]
DPCM vs. 2" slit vs. CH
[{Image src='images/factor/dpcm/DPCM_CH_2arcsec_only.gif}]
DPCM vs. 40" slit vs. CH
[{Image src='images/factor/dpcm/DPCM_CH_40arcsec_only.gif}]
DPCM vs. 266" slit vs. CH
[{Image src='images/factor/dpcm/DPCM_CH_266arcsec_only.gif}]
DPCM vs. 1" slit vs. FIL
[{Image src='images/factor/dpcm/DPCM_FIL_1arcsec_only.gif}]
DPCM vs. 2" slit vs. FIL
[{Image src='images/factor/dpcm/DPCM_FIL_2arcsec_only.gif}]
DPCM vs. 40" slit vs. FIL
[{Image src='images/factor/dpcm/DPCM_FIL_40arcsec_only.gif}]
DPCM vs. 266" slit vs. FIL
[{Image src='images/factor/dpcm/DPCM_FIL_266arcsec_only.gif}]
DPCM vs. 1" slit vs. PCH
[{Image src='images/factor/dpcm/DPCM_PCH_1arcsec_only.gif}]
DPCM vs. 2" slit vs. PCH
[{Image src='images/factor/dpcm/DPCM_PCH_2arcsec_only.gif}]
DPCM vs. 40" slit vs. PCH
[{Image src='images/factor/dpcm/DPCM_PCH_40arcsec_only.gif}]
DPCM vs. 266" slit vs. PCH
[{Image src='images/factor/dpcm/DPCM_PCH_266arcsec_only.gif}]
%%(text-align:center)
!DPCM compression vs different exposures
%%information
Compensation has been applied for those paused (and aborted) rasters. This is implemented by comparing %%(color:#cc0000;)NEXP%% and %%(color:#cc0000;)RAST_REQ%% and timing the ratio of two values.
At line 103 added 2 lines
%%information
There is a structure array to store all related information for the EIS data investigated here. The array has element with the following format:
At line 185 changed one line
DPCM vs. 10 second
{{{
compFactor={compression_factor, $
study_ACR :'', $ ;string
study_id :'', $ ;string
rast_ACR :'', $ ;string
rast_id :'', $ ;string
ll_ACR :'',$ ;string
ll_id :'',$ ;string
start_time :'', $ ;string
end_time :'', $ ;string
fitsname :'',$ ;string
target :'',$ ;string
sci_obj :'',$ ;string
slit :'',$ ;string
def_volume :0LL,$ ;long64 int, unit: bits
mdp_volume :0.0,$ ;float, unit: kbits
comp_scheme :0,$ ;int
nexp :0,$ ;int
rast_req :0,$ ;int
exposures :fltarr(8) $ ;float, unit: sec
}
}}}
At line 187 changed one line
[{Image src='images/factor/dpcm/DPCM_10s_only.gif}]
I attached an IDL sav file [here|https://vsolar.mssl.ucl.ac.uk/eiswiki/images/newgifs/20071201_20071215.sav.tar.gz]. You may download and play it, for example, I use:
At line 189 changed one line
DPCM vs. 15 second
{{{if (str1[i].SCI_OBJ eq 'QS') && (str1[i].COMP_SCHEME eq 1) && (str1[i].MDP_VOLUME gt 0.) then ind[i]=1}}}
At line 191 changed one line
[{Image src='images/factor/dpcm/DPCM_15s_only.gif}]
to extract records associated with 'QS' SCI_OBJ and using DPCM compression scheme.
----
At line 193 removed 158 lines
DPCM vs. 20 second
[{Image src='images/factor/dpcm/DPCM_20s_only.gif}]
DPCM vs. 30 second
[{Image src='images/factor/dpcm/DPCM_30s_only.gif}]
DPCM vs. 60s second
[{Image src='images/factor/dpcm/DPCM_60s_only.gif}]
%%(text-align:center)
!DPCM compression vs 1" SLA selection vs exposures
%%
DPCM vs. 1" slit vs. 10s exposures
[{Image src='images/factor/dpcm/DPCM_1arcsec_10s_only.gif}]
DPCM vs. 1" slit vs. 15s exposures
[{Image src='images/factor/dpcm/DPCM_1arcsec_15s_only.gif}]
DPCM vs. 1" slit vs. 20s exposures
[{Image src='images/factor/dpcm/DPCM_1arcsec_20s_only.gif}]
DPCM vs. 1" slit vs. 30s exposures
[{Image src='images/factor/dpcm/DPCM_1arcsec_30s_only.gif}]
DPCM vs. 1" slit vs. 60s exposures
[{Image src='images/factor/dpcm/DPCM_1arcsec_60s_only.gif}]
%%(text-align:center)
!DPCM compression vs 2" SLA selection vs exposures
%%
DPCM vs. 2" slit vs. 10s exposures
[{Image src='images/factor/dpcm/DPCM_2arcsec_10s_only.gif}]
DPCM vs. 2" slit vs. 15s exposures
[{Image src='images/factor/dpcm/DPCM_2arcsec_15s_only.gif}]
DPCM vs. 2" slit vs. 20s exposures
[{Image src='images/factor/dpcm/DPCM_2arcsec_20s_only.gif}]
DPCM vs. 2" slit vs. 30s exposures
[{Image src='images/factor/dpcm/DPCM_2arcsec_30s_only.gif}]
DPCM vs. 2" slit vs. 60s exposures
[{Image src='images/factor/dpcm/DPCM_2arcsec_60s_only.gif}]
%%(text-align:center)
!DPCM compression vs 40" SLA selection vs exposures
%%
DPCM vs. 40" slit vs. 10s exposures
[{Image src='images/factor/dpcm/DPCM_40arcsec_10s_only.gif}]
DPCM vs. 40" slit vs. 15s exposures
[{Image src='images/factor/dpcm/DPCM_40arcsec_15s_only.gif}]
DPCM vs. 40" slit vs. 20s exposures
[{Image src='images/factor/dpcm/DPCM_40arcsec_20s_only.gif}]
DPCM vs. 40" slit vs. 30s exposures
[{Image src='images/factor/dpcm/DPCM_40arcsec_30s_only.gif}]
DPCM vs. 40" slit vs. 60s exposures
[{Image src='images/factor/dpcm/DPCM_40arcsec_60s_only.gif}]
%%(text-align:center)
!DPCM compression vs 266" SLA selection vs exposures
%%
DPCM vs. 266" slit vs. 10s exposures
[{Image src='images/factor/dpcm/DPCM_266arcsec_10s_only.gif}]
DPCM vs. 266" slit vs. 15s exposures
[{Image src='images/factor/dpcm/DPCM_266arcsec_15s_only.gif}]
DPCM vs. 266" slit vs. 20s exposures
[{Image src='images/factor/dpcm/DPCM_266arcsec_20s_only.gif}]
DPCM vs. 266" slit vs. 30s exposures
[{Image src='images/factor/dpcm/DPCM_266arcsec_30s_only.gif}]
DPCM vs. 266" slit vs. 60s exposures
[{Image src='images/factor/dpcm/DPCM_266arcsec_60s_only.gif}]
%%
%%tab-SecondTab
This is the second tab
%%
%%tab-ThirdTab
This is the third tab
%%
%%
[{Image src='images/factor/75/JPEG75_only.gif'}]
QS QS,DC QS,SYN QS,CH QS,NET QS,DC,XBP QS,SYN,DC QS,DC,SEI QS,PCH
XBP,QS
AR AR,LMB AR,EFR AR,PLG AR,CH AR,NFL AR,CH,LMB
SYN SYN,DC SYN,QS,DC
PCH PCH,LMB,JET PCH,JET
JET,LMB,PCH
LMB LMB,LO LMB,PCH,SPI LMB,SPI
ENG,LMB,JET,PCH ENG,QS,DC
SYN,DC,QS SYN,DC
CH CH,JET CH,XBP CH,DC CH,LMB CH,QS CH,XBP,JET,EFR CH,BP,JET
PR PR,LMB PP,PCH
SPI,LMB
EFR SEI,EFR
DC DC,LMB WAV NET
SPI SPI,CH
BP,CH,JET BP,JET
FIL,PR FIL,QS
This page (revision-43) was last changed on 18-Apr-2021 19:03 by JianSunTop