## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Heart Rate Variability Analysis

Nathalie Claes

Tiensestraat 102, box 3726 3000 Leuven (Belgium) Nathalie.claes@ppw.kuleuven.be +003216/325885 2013 32

Content

Introduction ....................................................................................................................... 5 1 Analyzing Heartratevariability with ARTiiFACT ................................................... 7 1.1 1.2 1.3 1.4 2 Step 1: Extract IBIs from ECG data ................................................................ 8 Step 2: Detect and Process artifacts in IBIdata ............................................ 11 Step 3: Analyse HRV .................................................................................... 13 Measures of HRV ......................................................................................... 14

Interrater reliability ............................................................................................... 15 2.1 2.2 Importing your data in SPSS 20.0 ................................................................ 15 Calculating the interrater reliability with SPSS 20.0 ..................................... 15

3

Outlier analysis of HRV ........................................................................................ 17 3.1 3.2 3.3 3.4 3.5 Importing your data in SPSS 20.0 ................................................................ 17 Visual Inspection of data .............................................................................. 17 Normalizing variables ................................................................................... 19 Checking for outliers with SPSS 20.0 and Microsoft Excel 2010. ............... 20 Outlier exclusion criteria ............................................................................... 21

4

Reporting HRV measures .................................................................................... 21

CONTENT | 3

Introduction

In this paper, the analysis of HRV with ARTiiFACT is discussed. Another topic that is handled in this paper, is the detection of HRV outliers and what to do with them By no means this paper means to provide “the” way of conducting the analysis, but merely presents one way of handling your data. When conducting research, adequate and correct datahandling is very important, because it can have serious effects on your results. When using heart rate variability (HRV) data, one has to pay even more close attention to appropriate data analysis of the data, since HRV measures are very sensitive to artifacts. Handling artifacts correctly, is therefore crucial. One program to analyze HRV, is ARTiiFACT (see Kaufmann, Sütterlin, Schulz, & 2 Vögele, 2011 ). Video tutorials for this program are freely available on their website. The programs used to do analysis on the measures (outlieranalysis), are EXCEL 2010 and SPSS20.0. The methods described here however should also be similar in previous versions of SPSS and EXCEL. Good luck!

1

1

ARTiiFACT is a free to use software package developed by Kaufmann, Sütterlin, Schulz and Vögele. More information can be found on their website: http://www.artiifact.de Kauffman, T., Sütterlin, S., Schulz, S.M., & Vögele, C.(2011). ARTiiFACT: a tool for heart rate artifact processing and heart rate variability analysis. Behavior research methods, 43(4), 1161-1170. Doi: 10.3758/s13458-011-0107-7

2

INTRODUCTION | 5

1

Analyzing Heartratevariability with ARTiiFACT

After installing ARTiiFACT on your computer, just open it by double-clicking on the icon. A start-up screen will appear (as shown in Figure 1).

Figure 1. ARTiiFACT start screen.

One important tip I can give you is to define a working directory before starting the analysis. When you do that, all the files you create will be saved there. The first option, File conversion, can be used when your data samples are too big and you want to decrease the sample rate. It usually makes the processing go faster. This is however not necessary. The flow of events is as follows: first you complete the step “Extract IBIs from ECG data”(1) , next you “Detect and process artifacts in IBI data” (2) and lastly, yo u do the “Analyse HRV”(3). The Test for normal distribution is again optional and as the name says, it checks whether the HRV is normally distributed. Some steps have two options: file by file and batch. For step 1 and 2 it is recommended to use the file by file step, for the third step, the batch option can be very helpful, cf. infra. We discuss step 1, 2 and 3 separately.

ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT | 7

1.1

Step 1: Extract IBIs from ECG data

When clicking on the first step, the screen as shown in Figure 2 appears. You upload a file by selecting the folder icon. Make sure that there are no variable names in your data file, otherwise the program will not open your file.

Figure 2. ECG Preparation Module.

When you have selected a file, a window appears asking you to specify the number of header lines to ignore (should be 0), the samplingrate (Hz) that is dependent on your recording and downsampling, but typically it is 1000 (or 500 or 250, if you downsize it), and the channel that you want to process, typically 1.

Figure 3. Specifying options.

When you have specified everything, your ECG data should appear on screen and you have the option to cut out data that is contaminated. Data that deviates from the whole set (“peaks”) should be excluded. By clicking the select button in the right bottom of the screen and clicking the picture of the ECG signal, you can see the specific time of that data. For example, in the data as shown in Figure 4, the first 19 seconds appear to be noise, therefore we want to exclude them. We fill in what we wish to cut out and click apply. The area you want to exclude should color red. If you have more than one area that contains noise, you just repeat the procedure until every area you want to exclude is red. Click proceed.

8 | ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT

Figure 4. Data of a participant and excluding data.

Now the second phase of the first step can start. A new screen will appear, named “Automated Peak Detection Module” (See Figure 5). Here you see the ECG signal of your participant on the right. On the right you have several options to help you detect the R peaks. To inspect your data more closely, you can click the magnifying glass with a plus on the left and click on one of the lines. The invert ECG data should only happen if you notice that the peaks are inversed: the negative side of the peak is longer than the positive (up) side. A filter can be used to better select the peaks, either low or high pass. Most of the time, the high pass is used, and frequency is defined as 0.1 Hz, but this value can differ. In the example, it was 10. Also the peak detection method can be selected. If there were no trouble with your data collection, the global threshold option will do. To define the threshold, you can click the graph on the bottom and select the global peak level there. It can differ between participants. Also, the distribution of the R-R intervals are shown in a small graph on the left. When your filter settings are right, all the peaks (and some noise) will be marked in blue. When ready, click proceed.

ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT | 9

Figure 5. Automated Peak Detection Module.

A new screen will appear, with the possibility to manually select or deselect peaks. Accurate peak detection is important! For example, in Figure 6 is an area with a lot of noise, and you cannot know which is a peak and which is not. Therefore, you click the select button, select the peaks you want to mark as non-peaks (hold down alt when selecting multiple peaks). When you have selected all, you click the “mark as non-peak” button. When the program does not detect a peak it should detect, you can select it and push the mark as peak button. When ready, proceed.

Figure 6. Manual Peak Detection Module.

10 | ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT

A screen will appear (see Figure 7) showing you the distribution. You can give a name to the outputfile and choose to save it in excel, txt and/or pdf. Click save and quit to continue to the next step.

Figure 7. Store IBI module.

1.2

Step 2: Detect and Process artifacts in IBIdata

Now you have finished the first step, you can detect and process the artifacts in the IBIdata. Select the file by file option. Select the file that you created in the previous step. Anew, you have the option to cut away data, as you can see in Figure 8, some points around 100 ms and 300 ms could be deleted because they deviate too much from the rest from the sample. If you are insecure, whether they are artifacts or not, just proceed. In the next step, the program will tell you which it considers artifacts.

Figure 8. IBI Data Preparation Module.

ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT | 11

Next, the program shows you if there are artifacts in your dataset and indicates them in red. It is best to check the data yourself, because sometimes the program doesn‟t detect artifacts it should detect, or it falsely indicates something as an artifact that is not (as shown in Figure 9). An artifact is usually very visible: it deviates from the rest of your data, either upward or downward. Also, when two datapoints right after eachother are on equal height, it can be considered as an artifact: how big is the chance that two consecutive intervals are exactly the same size? To select a datapoint, first click the select option and next click the datapoint(s). Hold down alt to select multiple options. To mark it as an artifact or as valid data, click the respective button. Proceed.

Figure 9. Detecting Artifacts.

In the last step of step 2 (see Figure 10), you can choose a processing method of handling the artifacts. The method most used is the cubic spine interpolation. Click the refresh button next to the select method to run it. When it is done, the graphs should appear. Give your file a name and save it as a txt-file, an excel-file or/and an RSAtoolboxexport. Save and quit. Repeat step 1 and 2 for every participant.

12 | ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT

Figure 10. Interpolating data.

1.3

Step 3: Analyse HRV

When you completed step 1 and 2 for every participant, first create a new map where you put the files created in step 2 for every participant. Specify the file as your working directory. Next, click the batch option. Specify the map you just made and select in what format your outputfile should be saved. It is advisable to select only excel, since we will import the ratings in SPPS 20.0 and pdf-files will slow down the process. Click the refresh button on the bottom to start processing. The program shows you its progress and tells you when it is done. Your results can be found in the map you specified, for each participant separately as well as a summary file. In the summary file, participants are identified by the file name.

Figure 11. Analysis of HRV.

ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT | 13

1.4

Measures of HRV

The HRV measures that you will get out of the program are divided in a Time and a Frequency Domain. In table 1 you can find a definition of every measure that you will find in the output. Table 1. Overview of HRV measures Measure Time Domain Mean RR Median RR Mean HR SDNN RMSSD NN50 pNN50 Frequency Domain VLF (%) LF (%) HF (%) LF/HF 2 VLF (ms ) 2 LF (ms ) 2 HF (ms ) LF (n.u.) HF (n.u.) Definition

mean of interbeat (R-R) intervals median of interbeat (R-R) intervals mean of heartrate standard deviation of interbeat (R-R) intervals Root mean square of successive interbeat (R-R) intervals number of interbeat (R-R) intervals that differ more than 50 ms percentage interbeat (R-R) intervals that differ more than 50 ms

percentage of very-low-frequency component percentage low-frequency component percentage high-frequency component ratio of low and high frequency 2 very-low frequency component in ms 2 low frequency component in ms 2 high frequency component in ms normalized units of low frequency component normalized units of high frequency component

Some general remarks must be made about the different measures, however. The RMSSD for example is widely used, because it is a very precise measure, but it is also very sensitive to the experimental conditions. The NN50 is a rarely used measure, instead the Pnn50 is used (see TaskForce, 1996; and Ellen & Chambers, 2007). The LF is a depiction of both sympathetic and parasympathetic activity and is not easy to interpret. Therefore, it is not often used as a measure, but it is mentioned in the descriptive table. When you normalize the LF, it is closest as you can get to a measure of sympathetic activation. The HF on the other hand is a breathing-related measure that is very precise. However, it is also a very sensitive measure. When normalized, it is very dependent on the LF.

14 | ANALYZING HEARTRATEVARIABILITY WITH ARTIIFACT

2

Interrater reliability

Since artifacts can influence the HRV measures in various ways that contaminate your results, it is best to do the HRV analysis with two independent people. Next, rename the variables so that you can match them, e.g.: RMSSD1 for one rater, RMSSD2 for the other rater and add the files together in one excel-file.

2.1

Importing your data in SPSS 20.0

When all your measures are in one file, you can open the file in SPSS 20.0. To do so, you go to File → Open data and you will need to select the correct file, as shown in the figure below.

Figure 12. Opening a datafile in SPSS.

2.2

Calculating the interrater reliability with SPSS 20.0

To calculate the interrater reliability of the two raters, you go to Analyze → Scale → Reliability analysis. Do not forget to select the options you want in the statistics pane. Another way is to use syntax. Use the RELIABILITY command and specify which variables need to be compared and the model and scale you want to use. In this example, the interclass correlation coefficient (ICC) is calculated. An example:

INTERRATER RELIABILITY | 15

DATASET ACTIVATE DataSet1. RELIABILITY /VARIABLES= RMSSD1 RMSSD2 /SCALE('ALL VARIABLES') ALL /MODEL=ALPHA /STATISTICS=CORR ANOVA /ICC=MODEL(MIXED) TYPE(CONSISTENCY) CIN=95 TESTVAL=0. The output will look like this:

In the example above, the intraclass correlation is .85, and the F-test is significant. You also get a chronbach‟s alpha, but this is a measure of internal consistency. For interrater variability, it is better to report the ICC. The ICC value lies in between 0 and 1. The lower the ICC, the more variation in the method of judgment.

16 | INTERRATER RELIABILITY

3

Outlier analysis of HRV

Before using your data to test your hypothesis, it is best to check for outliers in Heart Rate Variability. There is no perfect way to check for HRV outliers, but it is important to be transparent about the method you used. The method described here is just one of many methods, and one can argue to use another method. Here, we first import our HRV data in SPSS 20.0, and subsequently delete data from participants that will be excluded from the experiment. Next, we inspect the data visually, by means of boxplots, histrograms and QQplots. Also, normality of distribution is checked and if need be, the measures which are skewed are normalized. When there are outliers in your file, decisions need to be made on the in- or exclusion of the specific outlier.

3.1

Importing your data in SPSS 20.0

Import your datafile with all the participants in SPSS 20.0 as descriped in 2.1. Next, recode the first variable so that you can identify your participants the same way as in your experiment dataset, e.g. “ID”. Sort it ascending. If you have to exclude participants (based on exclusion criteria, technical failures, etc.), do that before you start the inspection of your data.

3.2

Visual Inspection of data

A first step in detecting outliers in your HRV data, is to visually inspect your data. To inspect your data, go to Analyze → Descriptives → Explore. A screen will appear (see Figure 13). Put every variable you want to inspect in the dependent list. To know which cases are outliers, best put your identifier in the label cases by option. On the right, you have the possibility to specify statistics, plots and options. In the statistics pane, you can select that you want to see outliers. In the plots option, you can specify that you want normality tests.

OUTLIER ANALYSIS OF HRV | 17

Figure 13. Exploring your data.

Another option is use the EXAMINE variables command in your syntax. In ID you put the variable you want to label your cases with. In the PLOT command you indicate the graphs you want in your output, the same is true for the statistics command. An example: EXAMINE VARIABLES=MEANRR /ID=PPN /PLOT BOXPLOT STEMLEAF HISTOGRAM NPPLOT /COMPARE GROUPS /STATISTICS DESCRIPTIVES EXTREME /CINTERVAL 95 /MISSING LISTWISE /NOTOTAL. Regardless of the method you use, you get a table with descriptives, extreme values, and a table with tests of Normality. Often the Kolmogorov-Smirnov is used, and indeed, it can give you an indication of normality. Another way of checking if the distribution for your variable is normal, you can check the Normal Q-Q Plot. As you can see in Figure 14, the distribution of the variable there is quite normal, but you have an outlier on top.

Figure 14. Normal Q-Q Plot of variable MEANRR.

18 | OUTLIER ANALYSIS OF HRV

Another option to check a normal distribution is inspect the histograms and draw a normal distribution line in your graph. To confirm if you have an outlier, you can next inspect the boxplot of the respective variable. And indeed, as shown in Figure 15, meanRR has one outlier, more specifically participant 117. Whether you should exclude the participant or not, is dependent on a number of things, cf. infra.

Figure 15. Boxplot of variable MEANRR.

3.3

Normalizing variables

Some HRV measures are sensitive and are often skewed, for example: pNN50, HFabs, and RMSSD. Before you do any calculations with them, it is best to normalize the variables that are skewed in your specific dataset. To normalize the variables, a log transformation is often used, like for example the natural logarithmic transformation (ln) or the log10 transformation. To calculate the log transformation, go to Transform → Compute Variable. Give the new variable a name and use the LN(var) or Log10(var) option, e.g. COMPUTE LogRMSSD=LN(RMSSD). After you did that for all the variables that needed normalizing, check the normality distributions and outliers again for these variables.

OUTLIER ANALYSIS OF HRV | 19

3.4

Checking for outliers with SPSS 20.0 and Microsoft Excel 2010.

To check for outliers, you first calculate the mean and standard deviation for every HRV variable. Go to Analyze→ Descriptives → Descriptives. Or use a syntax, e.g: DESCRIPTIVES VARIABLES=MEANRR MEDIANRR MEANHR SDNN RMSSD NN50 Pnn50 VLF LF HF LFHF LFn.u HFn.u VLFabs LFabs HFabs LogRMSSD LogPNN50 LogNN50 LogLFHF LogLFabs LogVLFabs LogHFabs /STATISTICS=MEAN STDDEV. Next, decide on a criterion for exclusion of outliers. Common is the “mean ± 2/2.5/3 SD” criterion. The conservativeness of your criterion is an unavoidably subjective choice. One remark here is that you should be consistent in the criterion you use for all your data, or have good, justifiable reasons to do otherwise. In the example, 3 SD deviation from the mean is used as an criterion. To calculate the range of your data, click on the table with descriptives, and click export and export it to Excel. Open your excel-file and select a new column next to the table. In the first column you put in a formula to calculate the Mean – 3 SD, in the second column Mean + 3 SD option. For example, when for the variable MeanRR, the mean is in cell D3 and the SD is in cell E3, the formula is “=D3 – (3*E3) for the lower bound. Drag to extrapolate the formula for all the variables. Subsequently, you can create new variables that indicate if a case is an outlier (1) when it is not within the range of the mean SD or not (0). To do so, you recode the variable in a new variable as follows:

RECODE MeanRR(378.03 thru 1227.4 = 0) (ELSE=1) INTO OutlierMeanRR. EXECUTE. To see if you have outliers and which cases are variables, you go to the Analyze → Descriptives → Explore pane. Select the option to see the highest and lowest values. Or use a syntax: EXAMINE VARIABLES=OutlierMeanRR OutlierMedianRR OutlierMeanHR OutlierSDNN /ID=PPN /PLOT NONE /STATISTICS DESCRIPTIVES EXTREME /CINTERVAL 95 /MISSING LISTWISE /NOTOTAL.

20 | OUTLIER ANALYSIS OF HRV

It is possible that SPSS warns you he cannot show you a boxplot, since the variable is “constant”. This means that for that variable you have no outliers. For the other variables, check the table with „highest‟ values. Every case with a 1 shoul d be there. These are outliers for that variable.

3.5

Outlier exclusion criteria

Now you know which cases are outliers for specific variables, it is time to decide what to do with the outliers. The criteria/rules described here is just one method, which is not perfect. The most important aspects of making decisions are (a) being consistent in what you do, and (b) discuss your results with a colleague to have good interrater reliability of analysis decisions. When detecting outliers, you first have the criterion of the mean +- of 3 SD. Look at each outlier separately. When you have an outlier in meanRR, first check the original files for artifacts. If it is an artifact, redo the analysis and use the new information. If it is however not an artifact, exclude the whole case. If there is an outlier in the Time domain (cf. supra), we opt to exclude the whole case. If there is an outlier in the Frequency domain (cf. supra), like for example in LFabs or HFabs, we would opt to exclude only that specific variable, but not the Pnn50, because that is a very robust measure. When in doubt, you can always delete the complete variable or consult colleagues.

4

Reporting HRV measures

When collecting HRV data and reporting it, it is important to report (a) the method and program of collection and analysis, (b) the handling of artifacts, (c) the HRV measures used, (d) test of normality, (e) which variables were normalized, (f) if there were outliers and what happened to them. Also, a descriptive table should be included in your report, with the mean, standard deviation, a range or confidence intervals for the HRV measures, both the original as the normalized values. Make this descriptive table after you did the exclusions, on the included sample.

REPORTING HRV MEASURES | 21

RESEARCH GROUP ON HEALTH PSYCHOLOGY Tiensestraat 102 box 3726 3000 LEUVEN, BELGIË tel. + 32 16 32 58 85 nathalie.claes@ppw.kuleuven.be http://ppw.kuleuven.be/english/ogp

- case study report editedapi-270322257
- The Assessment of Fidelity in a Motor Speech Treatment ApproachPaulinaGutiérrezArias
- Smither performance.pdfbecbellary
- The Modified ADBB. Assessing for Infant Withdrawal During Routine ExaminationsAna Paula
- Paco Perez Studygdlo72
- science research final paperapi-274830241
- Report Data MiningSamoha Samoha
- ATQ_1.docxDanielle Marie Gevaña
- 2012 - Quality Control and Due Diligence in Project Management - Getting Decisions Right by Taking the Outside ViewDanielCaldeira
- Andreasen 1979 - TLCSivaprakash Balasundaram
- ATQ_1.docxDanielle Marie Gevaña
- ch18ceice2013
- 2003 LATERAL SHIFTLissette Cabezas Osorio
- 1757-1146-2-10karachiet
- slh-18-30Dino Alfonso Muñoz Cares
- Brown 2010Franco Medina C.
- Six Sigma in Manufacturing of Ingot Moulds in Foundry and Pattern Shop by Improving Sand QualityAJER JOURNAL
- 684Bolota Roxana Maria
- Apego Adulto Inseguro y Procesos Narrativos en PsicoterapiaJose
- The Applicability of the Local Government TheoriesHariyansaArip
- A Clinical Method for Identifying Scapular Dyskinesisalexjmatus
- Self-Dislcosure and LikeabilityAdrian Nathan West
- leeerAlfonso Hernández Ríos
- Andreasen (1979) Thought, language and communication disordersAbel González Luna
- Additional Psych Ass ReviewerBianca Marianne Uy
- Literatur for NdiNahdatunnisa Arumahi
- Am J Crit Care-2006-Gélinas-420-7Citra Indah Fitriwati
- MET_Feedback-for-Better-Teaching_Principles-Paper.pdfQueenie Butalid
- Narrative Change in Emotion-focused Therapy: How is Change Constructed Through the Lens of the Innovative Moments Coding System?EFTcouples
- 18598875Simion Bogdan

- SAJ_8_2011_26-37Sunway University
- UT Dallas Syllabus for stat6390.501.08s taught by Robert Serfling (serfling)UT Dallas Provost's Technology Group
- tmpDEC5.tmpFrontiers
- An enhanced Method for performing clustering and detecting outliers using mapreduce in dataminingInternational Journal for Scientific Research and Development
- Analysis And Detection of Infected Fruit Part Using Improved k-means Clustering and Segmentation TechniquesInternational Journal for Scientific Research and Development
- Pacific-Power--David-J-GodfreyGenability
- tmp31E0.tmpFrontiers
- Weighted Density based Error Optimization for Classical DatasetInternational Journal for Scientific Research and Development
- Multiple Linear Regression Models in Outlier DetectionWhite Globe Publications (IJORCS)
- Loan-to-Value Policy as a Macroprudential Tool: The Case of Residential Mortgage Loans in AsiaADBI Publications

- Business Statistics Level 3/Series 4 2008 (3009)Hein Linn Kyaw
- frbsf_let_19940114.pdfFRASER: Federal Reserve Archive
- 59916_1985-1989FRASER: Federal Reserve Archive
- IQS Brief - The New January Effectzerohedge
- Business Statstics/Series-4-2011(Code3009)Hein Linn Kyaw
- tmp4E81Frontiers
- Business Statistics/Series-2-2004(Code3009)Hein Linn Kyaw
- UT Dallas Syllabus for ba4368.001.11f taught by Kannan Ramanathan (kxr087000)UT Dallas Provost's Technology Group
- Business Statistics/Series-3-2010(Code3009)Hein Linn Kyaw
- maul-credo-response.pdfNational Education Policy Center
- Joseph Clemmie Moultrie v. Joseph R. Martin, Warden, 690 F.2d 1078, 4th Cir. (1982)Scribd Government Docs
- Rb LatimesiiNational Education Policy Center
- Fomc 19981117 MaterialFRASER: Federal Reserve Archive
- econcomm_19861101.pdfFRASER: Federal Reserve Archive
- The Persistence of Current Account Balances and its Determinants: The Implications for Global RebalancingADBI Publications
- tmpAF07.tmpFrontiers
- tmp6C25Frontiers
- Kelly Elementary School findingsLas Vegas Review-Journal
- 59915_1985-1989FRASER: Federal Reserve Archive
- UT Dallas Syllabus for ba4368.001.10f taught by Kannan Ramanathan (kxr087000)UT Dallas Provost's Technology Group
- tmpEF6E.tmpFrontiers
- Business Statistics Level 3/Series 3 2008 (Code 3009)Hein Linn Kyaw
- Sharyn Stagi v. Natl RR Passenger, 3rd Cir. (2010)Scribd Government Docs
- 32 Fair empl.prac.cas. (Bna) 708, 32 Empl. Prac. Dec. P 33,759, 13 Fed. R. Evid. Serv. 684 Equal Employment Opportunity Commission v. Western Electric Company, Incorporated, a Corporation, 713 F.2d 1011, 4th Cir. (1983)Scribd Government Docs
- Business Statistics/Series-2-2005(Code3009)Hein Linn Kyaw
- tmp86FDFrontiers
- Productivity Improvement using DMAIC: A Service Industry Case StudyInternational Journal for Scientific Research and Development
- 30 Fair empl.prac.cas. 906, 29 Empl. Prac. Dec. P 32,720 Equal Employment Opportunity Commission v. American National Bank, 680 F.2d 965, 4th Cir. (1982)Scribd Government Docs
- UT Dallas Syllabus for opre6364.501.11s taught by Kannan Ramanathan (kxr087000)UT Dallas Provost's Technology Group
- Nepc Ttr Folo Adler ResponseNational Education Policy Center

Sign up to vote on this title

UsefulNot usefulClose Dialog## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Loading