Prediction Modeling Methodology

  • Frank J. W. M. DankersEmail author
  • Alberto Traverso
  • Leonard Wee
  • Sander M. J. van Kuijk
Open Access

In the previous chapter, you have learned how to prepare your data before you start the process of generating a predictive model. In this chapter, you will learn how to make a predictive model using very common regression techniques and how to evaluate the performance of a model. In the next chapter we will then look at more advanced machine learning techniques that have become increasingly popular in recent years.

Hypothesis testing Null hypothesis p-value α level Confidence level Type I error Type II error Linear regression Logistic regression Bias-variance tradeoff Dimensionality reduction Feature selection Feature extraction Regularization Performance metrics R-squared Brier score Confusion matrix Prevalence Accuracy Positive predictive value Negative predictive value Sensitivity Specificity F1-score Receiver operating characteristic Area under the curve Model discrimination Model calibration Hosmer-Lemeshow test Calibration plot Internal validation External validation Random split Monte-Carlo cross-validation k-fold cross calibration Data leakage Model validation Cross-validation

8.1 Statistical Hypothesis Testing

A statistical hypothesis is a statement that can be tested by collecting data and making observations. Before you start data collection and perform your research, you need to formulate your hypothesis. An example hypothesis could be for instance: “If I increase the prescribed radiation dose to the tumor, this will also lead to an increase of side-effects in surrounding healthy tissues”. The purpose of statistical hypothesis testing is to find out whether the observations are meaningful or can be attributed to noise or chance.

The null hypothesis (often denoted H0) generally states the currently accepted fact. Often it is formulated in such a way that two measured values have no relation with each other. The alternative hypothesis, H1, states that there is in fact a relation between the two values. Rejecting or disproving the null hypothesis gives support to the belief that there is a relation between the two values.

To quantify the probability that a measured value originates from the distribution stated under the null statistical hypothesis tests are used that produce a p-value (e.g., Z-test or student’s t-test). The p-value gives the probability of obtaining a value equal to or greater than the observed value if the null hypothesis is true. A high p-value indicates that the observed value is likely under the null assumption, vice versa a low p-value indicates that the observed value is unlikely given the null hypothesis, which can lead to its rejection.

There are common misconceptions regarding the interpretation of the p-value [1]:

  • The p-value is not the probability of falsely rejecting the null hypothesis (type I error, see below)

  • A low p-value does not prove the alternative hypothesis

The p-value is to be used in combination with the α level. The α level is a predefined significance level by the researcher which equals the probability of falsely rejecting the null hypothesis if it is true (type I error). It is the probability (s)he deems acceptable for making a type I error. If the p-value is smaller than the α level, the result is said to be significant at the α level and the null hypothesis is rejected. Commonly used values for α are 0.05 or 0.001 (Fig. 8.1).
Fig. 8.1

Illustration of null hypothesis testing. The p-value represents the probability of obtaining a value equal or higher than the test value. The α level is predefined by the researcher and represents the accepted probability of making a type I error where the null hypothesis is falsely rejected. If the p-value of a statistical test is larger than the α level the null hypothesis is rejected

Confidence levels serve a similar purpose as the α level, and by definition the confidence level + α level = 1. So an α level of 0.05 corresponds to a 95% confidence level.

8.1.1 Types of Error

We distinguish between two types of errors in statistical testing [2]. If the null hypothesis is true but falsely rejected, this is called a type I error (comparable to a false positive, with a positive result indicating the rejection). The type I error rate, the probability of making a type I error, is equal to the α level since that is the significance level at which we reject the null hypothesis. Likewise, if the null hypothesis is false but not rejected, this is called a type II error (comparable to a false negative, with a negative result indicating the failed rejection) (Table 8.1).
Table 8.1

The two types of errors that can be made regarding the acceptance or rejection of the null hypothesis

Null hypothesis truth

Null hypothesis decision

Fail to reject

Type II error

(false negative)

Type I error

(false positive)

8.2 Creating a Prediction Model Using Regression Techniques

8.2.1 Prediction Modeling Using Linear and Logistic Regression

A prediction model tries to stratify patients for their probability of having a certain outcome. The model then allows you to identify patients that have an increased chance of an event and this may lead to treatment adaptations for the individual patient. For instance, if a patient has an increased chance of a tumor recurrence the doctor may opt for a more aggressive treatment, or, if a patient has a high risk of getting a side-effect a milder treatment might be indicated.

The outcome variable of the prediction model can be anything, e.g., the risk of getting a side effect, the chance of surviving at a certain time point, or the probability of having a tumor recurrence. We can distinguish outcome variables into continuous variables or categorical variables. Continuous variables are described by numerical values and regression models are used to predict them, e.g., linear regression . Categorical variables are restricted to a limited number of classes or categories and we use classification models for their prediction. If the outcome has two categories this is referred to as binary classification and typical techniques are decision trees and logistic regression (somewhat confusingly, this regression method is well suited for classification due to its function shape).

Fitting or training a linear or logistic prediction model is a matter of finding the function coefficients so that the model function optimally follows the data (Fig. 8.2).
Fig. 8.2

Examples of predictive modeling (blue line) for a continuous outcome using linear regression and for a binary outcome using logistic regression. The predictions in the logistic regression are rounded to either class A or B using a threshold (0.5 by default)

8.2.2 Software and Courses for Prediction Modeling

There are many different software packages available for generating prediction models, all of them with different advantages and disadvantages. Some packages are code-based and programming skills are required, e.g., Python, R or Matlab. There are integrated development environments available for improved productivity, like RStudio for R, and Spyder for Python. Additionally, they can have rich open-source libraries tailored specifically towards machine learning, for instance Caret for R [3] and Scikit-learn for Python [4]. Other packages have graphical user interfaces and being able to program is not mandatory, like SPSS, SAS or Orange. Some packages are only commercially available, but many are open-source and have a large user base for support.

Preference for a certain software package over others is very personal and the best advice is therefore to simply try several and find out for yourself. A special mention is reserved for the Anaconda Distribution [5], which hosts many of the most widely used software packages for prediction modeling in a single platform (RStudio, Spyder, Jupyter Notebook, Orange and more) (Table 8.2).
Table 8.2

A non-exhaustive overview of available software packages for prediction modeling and some of their features

Coding required

Development environments

Learn more (books/tutorials)

RStudio [7]

Spyder [10]

Jupyter notebooks [11]

Partly (students)

Visual workflows

Visual workflows

N/A not applicable

There is a wealth of freely available information on the Web to help you get going. Providing a comprehensive overview is therefore an impossible task, but some excellent online courses (sometimes referred to as Massive Open Online Courses or MOOCs) are listed below (Table 8.3).
Table 8.3

Free online courses for prediction modeling and machine learning

Machine learning

Andrew Ng, Stanford University, Coursera

Machine learning

Tom Mitchell, Carnegie Mellon University

Learning from data

Yaser Abu-Mostafa, California Institute of Technology

Machine learning

Nando de Freitas, University of Oxford

8.2.3 A Short Word on Modeling Time-to-Event Outcomes

Many studies are interested not only in predicting a certain outcome, but additionally take into account the time it takes for this outcome to occur. This is referred to as time-to-event analysis and a typical example is survival analysis. Kaplan-Meier curves are widely used for investigation of the influence of categorical variables [23], whereas Cox regression (or sometimes called Cox proportional hazards model) additionally allows the investigation of quantitative variables [24].

8.3 Creating a Model That Performs Well Outside the Training Set

8.3.1 The Bias-Variance Tradeoff

The bias-variance tradeoff explains the difficulty of a generated prediction model to generalize beyond the training set, i.e. perform well in an independent test set (also called the out-of-sample performance). The error of a model in an independent test set can be shown to be decomposable into a reducible component and an irreducible component. The irreducible component cannot be diminished, it will always be present no matter how good the model will be fitted to the training data. The origin of the irreducible error can, for instance, be an unmeasured but yet important variable for the outcome that is to be predicted.

The reducible error can be further decomposed into the error due to variance and the error due to bias [2]. The variance is the error due to the amount of overfitting done during model generation. If you use a very flexible algorithm, e.g., an advanced machine learning algorithm with lots of freedom to follow the data points in the training set very closely, this is more likely to overfit the data. The error in the training set will be small, but the error in the test set will be large. Another way to look at this is that a high variance will result in very different models during training if the model is fitted using different training sets.

Bias relates to the error due to the assumptions made by the algorithm that is chosen for model generation. If a linear algorithm is chosen, i.e. a linear relation between the inputs and the outcome is assumed, this may cause large errors (large bias) if the underlying true relation is far from linear. Algorithms that are more flexible (e.g., neural networks) result in less bias since they can match the underlying true but complex relations more closely.

In general it can be said that:
  • Flexible algorithms have low bias since they can more accurately match the underlying true relation, but have high variance since they are susceptible to overfitting.

  • Inflexible algorithms have low variance since they are less likely to overfit, but have high bias due to their problems of matching the underlying true relationship.

From this we can conclude that the final test set error is a tradeoff between low bias and low variance. It is impossible to simultaneously achieve the lowest possible variance and bias. The challenge is to generate a model with (reasonably) low variance and low bias since that is most likely to generalize well to external sets. This model might have slightly decreased performance in the training set, but will have the best performance in subsequent test sets (Fig. 8.3).
Fig. 8.3

The bias-variance tradeoff. With increased model complexity the model can more accurately match the underlying relation at the risk of increasing the variance (amount of overfitting). The bias-variance tradeoff corresponds to minimizing the total prediction error (which is the sum of bias and variance)

8.3.2 Techniques for Making a General Model

As we collect and expand our datasets we often score many features (parameters) so that we minimize the risk of potentially missing important features, i.e. features that are highly predictive of the outcome. This means that generally we deal with wide sets containing many features. However, many of these features are in fact redundant or not relevant for the outcome at all and can be safely omitted. Including a large number of features during model generation increases the possibility of chance correlations of features with the outcome (overfitting) and this results in models that do not generalize well. Dimensionality reduction [25], reducing the number of features, is therefore an important step prior to model generation. The main advantages are:
  • Lowered chance of overfitting and improved model generalization

  • Increased model interpretability (depending on the method of dimensionality reduction)

  • Faster computation times and reduced storage needs

There are many useful dimensionality reduction techniques. The first category of methods to consider is that of feature selection, where we limit ourselves to a subset of the most important features prior to model generation. Firstly, if a feature has a large fraction of missing values it is unlikely to be predictive of the outcome and can often be safely removed. In addition, if a feature has zero or near zero variance, i.e. its values are all highly similar, this again indicates that the feature is likely to be irrelevant. Another simple step is to investigate the inter-feature correlation, e.g., by calculating the Pearson or Spearman correlation matrix. Features that are highly correlated with each other are redundant for predicting the outcome (multicollinearity). Even though a group of highly correlated features may all be predictive of the outcome, it is sufficient to only select a single feature as the others provide no additional information.

Traditionally, further feature selection is then performed by applying stepwise regression. In each step a feature is either added or removed and a regression model is fitted and evaluated based on some selection criterion. There are many choices for the criterion to choose between models, e.g., the Bayesian information criterion or the Akaike information criterion, both of which quantify the measure of fit of models and additionally add a penalty term for complex models comprising more parameters [26]. In forward selection, one starts with no features and the feature that improves the model the most is added to the model. This process is repeated until no significant improvement is observed. In backward elimination, one starts with a model containing all features, and features are removed that decrease the model performance the least, until no features can be removed without significantly decreasing performance.

With feature selection we limit ourselves to a subset of features that are already present in the dataset and this is a special case of dimensionality reduction [27]. In feature extraction the number of features are reduced by replacing the existing features by fewer artificial features which are combinations of the existing features. Popular techniques for feature extraction are principle component analysis, linear discriminant analysis and autoencoders [25].

More advanced machine learning algorithms often contain embedded methods for reducing model complexity to improve generalizability. An example is regularization where each added feature also comes with an added penalty or cost [8]. The addition of a feature may increase the model performance but, if the added cost is too high, it will not be included in the final model. This effectively performs feature selection and prevents overfitting. The severity of the cost is a hyperparameter that can be tuned (e.g., through cross-validation, see paragraph “Techniques for internal validation”). Popular regularization methods for logistic regression are LASSO (or L1 regularization) [28], ridge (or L2 regularization), or a combination of both using Elastic Net [29]. The main difference between L1 and L2 regularization is that in L1 regularization the coefficients of unimportant parameters shrink to zero, effectively performing feature selection and simplifying the final model.

8.4 Model Performance Metrics

8.4.1 General Performance Metrics

The performance of a prediction model is evaluated by the calculation of performance metrics. We want our model to have high discriminative ability, i.e. high probabilities should be predicted for observations having positive classes (e.g., alive after 2 years or treatment) and low probabilities for negative classes (dead after 2 years of treatment). There is no general best performance metric for model evaluation as this depends strongly on the underlying data as well as the intended application of the model.

Other often-used overall performance metrics are R-squared measures of goodness of fit (or R2, also called the coefficient of determination). The R2 can be interpreted as the amount of variance in the data that is explained by the model (explained variation). Higher R2s correspond to better models. Examples are Cox and Snell’s R2 or Nagelkerke’s R2. R-squared values are mainly used in regression models; for classification models it is more appropriate to look at performance metrics derived from the confusion matrix.

Another popular overall performance measure is the Brier score (or mean squared error) and it is defined as the average of the square of the difference between the predictions and observations. A low Brier score indicates that predictions match observations and we are dealing with a good model.

8.4.2 Confusion Matrix

The confusion matrix is a very helpful tool in assessing model performance. It lists the correct and false predictions versus the actual observations and allows for the calculation of several insightful performance metrics. If the output of your prediction model is a probability (e.g., the output of a logistic regression model), then it needs to be dichotomized first by applying a threshold (typically 0.5) before the confusion matrix can be generated. An exemplary confusion matrix is shown in Table 8.4.
Table 8.4

Confusion matrix showing predictions and observations. Many useful performance metrics are derived from the values in the confusion matrix

True positive

False positive

Positive predictive value (PPV)

False negative

True negative

Negative predictive value

True positives, called hits, are cases that are correctly classified. True negatives are correctly rejected. False positives, or false alarm, are equivalent to a type I error. False negatives, or misses, are equivalent to a type II error.

Prevalence is defined as the number of positive observations with respect to the total observations. A balanced dataset has a prevalence close to 0.5, or 50%. Often, we have to deal with imbalanced datasets and this can lead to difficulties when interpreting certain performance metrics. Performance metrics can be high for poor models that are trained and tested on imbalanced datasets.
$$ \mathrm{Prevalence}=\left(\mathrm{TP}+\mathrm{FN}\right)/\left(\mathrm{TN}+\mathrm{TP}+\mathrm{FP}+\mathrm{FN}\right) $$

8.4.3 Performance Metrics Derived from the Confusion Matrix

Accuracy , defined as the proportion of correct predictions, is often reported in literature. Care has to be taken when using this metric in highly imbalanced datasets. Consider a dataset with only 10% positive observations. If the prediction model simply always predicts the negative class it will be correct in 90% of the cases. The accuracy is high, but the model is useless since it cannot detect any positive cases.
$$ \mathrm{Accuracy}=\left(\mathrm{TN}+\mathrm{TP}\right)/\left(\mathrm{TN}+\mathrm{TP}+\mathrm{FP}+\mathrm{FN}\right) $$
Another option is to look at the proportion of correct positive predictions for the total number of positive observations. This is called the Positive Predictive Value (PPV) or precision. Similarly, the proportion of correct negative predictions for the total number of negative observations is called the Negative Predictive Value (NPV), respectively. These metrics are of interest to patients and clinicians as they give the probability that the prediction matches the observation (truth) for a patient. PPV and NPV are dependent on the prevalence in the dataset making their interpretation more difficult. A high prevalence will increase PPV and decrease NPV (while keeping other factors constant).
$$ \mathrm{PPV}=\mathrm{TP}/\left(\mathrm{TP}+\mathrm{FP}\right) $$
$$ \mathrm{NPV}=\mathrm{TN}/\left(\mathrm{TN}+\mathrm{FN}\right) $$

If we want to consider characteristics not of the population but of the prediction model when applied as a clinical test, we can evaluate sensitivity and specificity. Sensitivity, or True Positive Rate (TPR, or sometimes called recall or probability of detection), is defined as the probability of the model to make a positive prediction for the entire group of positive observations. It is a measure of avoiding false negatives, i.e. not missing any diseased patients.

Similarly, specificity is defined as the probability of the model to make a negative prediction for the entire group of negative observations. It is a measure of avoiding false positives, i.e. not including non-diseased patients.
$$ \mathrm{TPR}=\mathrm{TP}/\left(\mathrm{TP}+\mathrm{FN}\right)\kern1.875em \left(\mathrm{sensitivity}\right) $$
$$ \mathrm{TNR}=\mathrm{TN}/\left(\mathrm{TN}+\mathrm{FP}\right)\kern1.625em \left(\mathrm{specificity}\right) $$
Additionally, we can determine the False Positive Rate (FPR), or fall-out, and the False Negative Rate (FNR), which are the opposites of TPR and FPR, respectively. Note that FNR is used in the next paragraph for the construction of the Receiver Operating Characteristic curve.
$$ \mathrm{FNR}=1-\mathrm{TPR}=1-\mathrm{sensitivity} $$
$$ \mathrm{FPR}=1-\mathrm{TNR}=1-\mathrm{specificity} $$
The F1-score , or F-score, is a metric combining both PPV (precision) and TPR (recall or sensitivity). Unlike PPV and TPR separately, it takes both false positives and false negatives into account simultaneously. It does however still omit the true negatives. It is typically used instead of accuracy in the case of severe class imbalance in the dataset.
$$ \mathrm{F}1=2\cdotp \left(\mathrm{PPV}\cdotp \mathrm{TPR}\right)/\left(\mathrm{PPV}+\mathrm{TPR}\right) $$

8.4.4 Model Discrimination: Receiver Operating Characteristic and Area Under the Curve

The performance of a prediction model is always a tradeoff between sensitivity and specificity. By changing the threshold that we apply to round our model predictions to positive or negative classes, we can change the sensitivity and specificity of our model. By decreasing this threshold, we are making it easier for the model to make positive predictions. The number of false negatives will go down but false positives will go up, increasing sensitivity but lowering specificity. By increasing the threshold, the model will make fewer positive predictions, the number of false negatives will go up and false positive will go down, decreasing sensitivity and increasing specificity (Fig. 8.4).
Fig. 8.4

Influence of the threshold that is used to round model prediction probabilities to 0 or 1. By using a low threshold the model will detect most of the patients with the outcome (high sensitivity), but many patients without the outcome will also be included (low specificity). For each value of the threshold sensitivity and specificity values can be calculated

By evaluating different thresholds for rounding our model predictions, we can determine many sensitivity and specificity pairs. If we plot the sensitivity versus (1 – specificity) for all these pairs, i.e. the true positive rate versus the false positive rate, we obtain the Receiver Operating Characteristic curve (ROC) [30]. This curve can give great insight into model discrimination performance. It allows for determining the optimal sensitivity/specificity pair of a model so that it can support decision making, and also allows comparison of different models with each other.

Powerful models have ROC curves that approach the upper left corner, which indicates that the model achieves the maximum of 100% sensitivity and 100% specificity simultaneously. Conversely, a poor model with no predictive value will have an ROC curve close to the y = x or 45 degree line. This has led to the use of the Area Under the Curve (AUC) of the ROC curve (or concordance statistic, c) as a widely used metric for interpreting individual model performance but also for comparing between models. Strong performing models have higher ROC curves and thus larger AUC values. A perfect model making correct predictions for every patient has an AUC of 1, whereas a useless model giving random predictions results in an AUC of 0.5. The AUC can be interpreted as the probability that the model will give a higher predicted probability to a randomly chosen positive patient than a randomly chosen negative patient (Fig. 8.5).
Fig. 8.5

ROC curve indicating discriminating performance of the model. Model predictions are rounded to 0 or 1 using many different thresholds resulting in the sensitivity and specificity pairs that form the curve. AUC is indicated by the gray area under the curve. Higher values correspond to better model discrimination performance

8.4.5 Model Calibration

Historically, the focus in evaluating model performance has primarily been on discriminative performance, e.g., by calculating R2 metrics, confusion matrix metrics and performing ROC/AUC analysis. Model calibration is however as important as discrimination and should always be evaluated and reported. Model calibration refers to the agreement between subgroups of predicted probabilities and their observed frequencies. For example, if we collect 100 patients for which our model predicts 10% chance of having the outcome, and we find that in reality 10 patients actually have the outcome, then our model is well calibrated. Since the predicted probabilities can drive decision-making it is clear that we want the predictions to match the observed frequencies.

A widely-used (but no so effective) way of determining model calibration is by performing the Hosmer-Lemeshow test for goodness of fit of logistic regression models. The test evaluates the correspondence between predictions and observations by dividing the probability range [0-1] into n subgroups. Typically, 10 subgroups are chosen, but this number is arbitrary and can have a big influence on the final p-value of the test.

A better approach is to generate a calibration plot [31, 32, 33]. It is constructed by ordering the predicted probabilities, dividing them into subgroups (again, typically 10 is chosen) and then plotting the average predicted probability versus the average outcome for each subgroup. The points should lie close to the ideal line of y = x indicating agreement between predictions and observed frequencies for each subgroup. Helpful additions are error bars, a trend line (often a LOESS smoother [34]), individual patient predictions versus outcomes and/or histograms of the distributions of positive and negative observations (the graph is then sometimes called a validation plot) (Fig. 8.6).
Fig. 8.6

Calibration plot indicating agreement between model predictions and observed frequencies. Data points are subdivided into groups for which the mean observation is plotted against the mean model probability. Perfect model calibration corresponds with the y = x line. Additionally, individual data points are shown (with some added y-jitter to make them more clear), as well as histograms for the positive and negative classes [0,1]

8.5 Validation of a Prediction Model

8.5.1 The Importance of Splitting Training/Test Sets

In the previous paragraphs different metrics for evaluation of model performance have been discussed. As briefly discussed in paragraph “The bias-variance tradeoff” it is important to compute performance metrics not on the training dataset but on data that was not seen during the generation of the model, i.e. a test or validation set. This will ensure that you are not mislead into thinking you have a good performing model, while it may in fact be heavily overfitted on the training data. Overfitting means that the model is trained too well on the training set and starts to follow the noise in the data. This generally happens if we allow too many parameters in the final model. The performance on the training set is good, but on new data the model will fail. Underfitting corresponds to models that are too simplistic and do not follow the underlying patterns in the data, again resulting in poor performance in unseen data.

Properly evaluating your model on new/unseen data will improve the generalizability of the model. We differentiate between internal validation, where the dataset is split into a training set for model generation and a test set for model validation, and external validation, where the complete dataset is used for model generation and separate/other datasets are available for model validation.

8.5.2 Techniques for Internal Validation

If you only have a single dataset available you can generate a test set by slicing of a piece of the training set. The simplest approach is to use a random split , e.g., using 70% of the data for training and 30% for evaluation (sometimes called a hold-out set). It is important to stratify the outcome over the two sets, i.e. make sure the prevalence in both sets remains the same. The problem with this method is that we can never be sure that the calculated performance metric is a realistic estimate of the model performance on new data or due to a(n) ‘(un)lucky’ randomization. This can be overcome by repeating for many iterations and averaging the performance metrics. This method is called Monte Carlo cross-validation (Fig.8.7) [35].
Fig. 8.7

Schematic overview of a Monte Carlo cross-validation. A random stratified split is applied to separate a test set from the training set. A prediction model is trained on the training set and performance metrics on the test set are stored after which the process is repeated

Another approach is the method of k-fold cross-validation [36]. In this method the data is split into k stratified folds. One of these folds is used as a test set, the others are combined and used for model training. We then iterate and use every fold as a test set once. A better estimate of the true model performance can be achieved by averaging the model performances on the test set. Typically, k = 5 or k = 10 is chosen for the number of folds (Fig. 8.8).
Fig. 8.8

Schematic overview of k-fold cross-validation. The dataset is randomly split into k stratified folds. Each fold is used as a test set once, while the other folds are temporarily combined to form a training set for model generation. Performance metrics on the test set are calculated and stored, and the process is repeated for the number of folds that have been generated

The advantage of k-fold cross-validation is that each data point is used in a test set only once, whereas in Monte Carlo cross-validation it can be selected multiple times (and other points are not selected for a test set at all), possibly introducing bias. The disadvantage of k-fold cross-validation is that it only evaluates a limited number of splits whereas Monte Carlo cross-validation evaluates as many split as you desire by increasing the number of iterations (although you could iterate the entire k-fold cross-validation procedure as well which is commonly called repeated k-fold cross-validation).

Note that in both Monte Carlo cross-validation and k-fold cross-validation we are generating many models instead of a single final model, e.g., because the feature selection algorithm might select different features or the regression produces different coefficients due to different training data. Cross-validation is used to identify the best method (i.e. data pre-processing, algorithm choice etc.) that is to be used to construct your final model. When you have identified the optimal method you can then train your model accordingly on all the available data.

A common mistake in any method where the dataset is split into training and test sets is to allow data leakage to occur [37]. This refers to using any data or information during model generation that is not part of the training set and can result in overfitting and overly optimistic model performance. It can happen for example when you do feature selection on the total dataset before applying the split. In general it is advised to perform any data pre-processing steps after the data has been split and using only information available in the training set.

8.5.3 External Validation

The true test of a prediction model is to evaluate its performance under external validation, or separate datasets from the training dataset. Preferably, this is performed on new data acquired from a different institution. It will indicate the generalizability of the model and show whether it is overfitted on the training data. If this can be performed on multiple external validation sets, this further strengthens the acceptance of the prediction model under evaluation.

It has to be noted that if the datasets intended to be used for external validation are collected by the same researchers that built the original prediction model, this is still not an independent validation. Independent external validation, by other researchers, is the ultimate test of the model generalizability. This requires open and transparent reporting of the prediction model, of inclusion and exclusion criteria for the training cohort and of data pre-processing steps. Additionally, it is encouraged to make the training data publicly available as this allows other researchers to verify your methodology and results and greatly improves reproducibility.

8.6 Summary Remarks

8.6.1 What Has Been Learnt

In this chapter you have learnt about the importance of the bias-variance tradeoff in prediction modeling applications. You have learnt how to generate a simple logistic regression model and what metrics are available to evaluate its performance. It is important to not limit the evaluation to model discrimination only, but also include calibration as well. Finally, we have discussed the importance of separating training and test sets so that we protect ourselves from overfitting. Internal validation strategies such as cross-validation are discussed, and the ultimate test of a prediction model, independent external validation, has been emphasized.

8.6.2 Further Reading

The field of prediction modeling and machine learning is extremely broad and in this chapter we have only scratched the surface. A good place to start with further reading on the many aspects of prediction modeling is the book “Clinical Prediction Models – A Practical Approach to Development, Validation, and Updating” by Steyerberg [38]. If you are looking to improve your knowledge and simultaneously improve your practical modeling skills the book “An Introduction to Statistical Learning – with Applications in R” by James et al. is highly recommended [8]. Finally, if you want to go in-depth and understand the underlying principles of the many machine learning algorithms the go-to book is “The Elements of Statistical Learning – Data Mining, Inference, and Prediction” by Hastie et al. [39].

  1. 1.
    Goodman S. A dirty dozen: twelve P-value misconceptions. Semin Hematol. 2008;45(3):135–40.CrossRefPubMedGoogle Scholar
  2. 2.
    Banerjee A, Chitnis UB, Jadhav SL, Bhawalkar JS, Chaudhury S. Hypothesis testing, type I and type II errors. Ind Psychiatry J. 2009;18(2):127–31.CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Kuhn M, et al. Caret: classification and regression training, 2016.Google Scholar
  4. 4.
    Pedregosa F, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825–30.Google Scholar
  5. 5.
    Anaconda distribution: The most popular Python/R data science distribution. [Online]. Available:
  6. 6.
    R: The R Project for Statistical Computing. [Online]. Available:
  7. 7.
    RStudio: Open source and enterprise-ready professional software for R, RStudio. [Online]. Available:
  8. 8.
    James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning – with applications in R. 1st ed. New York: Springer; 2013.CrossRefGoogle Scholar
  9. 9.
    Python: The official home of the Python Programming Language. [Online]. Available:
  10. 10.
    Spyder: The Scientific PYthon Development EnviRonment. [Online]. Available:
  11. 11.
    Jupyter: Open-source web application for live coding, data visualizations, numerical simulation, statistical modeling and more. [Online]. Available:
  12. 12.
    Müller A, Guido S. Introduction to machine learning with Python. Sebastopol: O’Reilly Media; 2016.Google Scholar
  13. 13.
    Matlab: The easiest and most productive software environment for engineers and scientists. [Online]. Available:
  14. 14.
    Murphy P. Machine learning: a probabilistic perspective. Cambridge: The MIT Press; 2012.Google Scholar
  15. 15.
    SPSS: The world’s leading statistical software used to solve business and research problems by means of ad-hoc analysis, hypothesis testing, geospatial analysis and predictive analytics. [Online]. Available:
  16. 16.
    George D, Mallery P. IBM SPSS statistics 23 step by step: Pearson Education; 2016.Google Scholar
  17. 17.
    SAS: SAS/STAT State-of-the-art statistical analysis software for making sound decisions. [Online]. Available:
  18. 18.
    SAS/STAT® 13.1 User’s Guide. SAS Institute Inc, 2013.Google Scholar
  19. 19.
    Orange: Open source machine learning and data visualization for novice and expert. [Online]. Available:
  20. 20.
    Weka: Data mining software in Java. [Online]. Available:
  21. 21.
    Witten I, Frank E, Hall M, Pal C. Data mining: practical machine learning tools and techniques. Burlington: Morgan Kaufmann; 2016.Google Scholar
  22. 22.
    RapidMiner Studio: Visual workflow designer for data scientists. [Online]. Available:
  23. 23.
    Efron B. Logistic regression, survival analysis, and the Kaplan-Meier curve. J Am Stat Assoc. 1988;83(402):414–25.CrossRefGoogle Scholar
  24. 24.
    Walters SJ. Analyzing time to event outcomes with a Cox regression model. Wiley Interdiscip Rev Comput Stat. 2012;4(3):310–5.CrossRefGoogle Scholar
  25. 25.
    van der Maaten LJP, Postma EO, van den Herik HJ. Dimensionality reduction: a comparative review. Tilburg University Technical Report TiCC TR 2009-005; 2009.Google Scholar
  26. 26.
    Vrieze SI. Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychol Methods. 2012;17(2):228–43.CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    Dash M, Liu H. Feature selection for classification. Intell Data Anal. 1997;1(3):131–56.CrossRefGoogle Scholar
  28. 28.
    Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Methodol. 1996;58(1):267–88.Google Scholar
  29. 29.
    Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Ser B Stat Methodol. 2005;67(2):301–20.CrossRefGoogle Scholar
  30. 30.
    Fawcett T. An introduction to ROC analysis. Pattern Recogn Lett. 2006;27(8):861–74.CrossRefGoogle Scholar
  31. 31.
    Steyerberg EW, et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology. 2010;21(1):128–38.CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Moons KGM, et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 2012;98(9):683–90.CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–31.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 33(3):517–35.CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Xu Q-S, Liang Y-Z. Monte Carlo cross validation. Chemom Intell Lab Syst. 2001;56(1):1–11.CrossRefGoogle Scholar
  36. 36.
    Rodriguez JD, Perez A, Lozano JA. Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans Pattern Anal Mach Intell. 2010;32(3):569–75.CrossRefPubMedGoogle Scholar
  37. 37.
    Luo W, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. 2016;18(12)CrossRefPubMedPubMedCentralGoogle Scholar
  38. 38.
    Steyerberg E. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer-Verlag; 2009.CrossRefGoogle Scholar
  39. 39.
    Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. New York: Springer-Verlag; 2001.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  • Frank J. W. M. Dankers
    • 1
    • 2
    Email author
  • Alberto Traverso
    • 1
  • Leonard Wee
    • 1
  • Sander M. J. van Kuijk
    • 3
  1. 1.Department of Radiation Oncology (MAASTRO), GROW School of Oncology and Developmental Biology (GROW)Maastricht University Medical CenterMaastrichtThe Netherlands
  2. 2.Department of Radiation OncologyRadboud University Medical CenterNijmegenThe Netherlands
  3. 3.Department of Clinical Epidemiology and Medical Technology Assessment (KEMTA)Maastricht University Medical CenterMaastrichtThe Netherlands

Personalised recommendations


Biography Duke Ellington

Duke Ellington
called his music 'American Music' rather than jazz, and liked to describe those who impressed him as 'beyond category. He remains one of the most influential figures in jazz, if not in all American music and is widely considered as one of the twentieth century's best known African American personalities. As both a composer and a band leader, Ellington's reputation has increased since his death, with thematic repackaging of his signature music often becoming best-sellers. Posthumous recognition of his work include a special award citation from the Pulitzer Prize Board.

Simply put, Ellington transcends boundaries and fills the world with a treasure trove of music that renews itself through every generation of fans and music-lovers. His legacy continues to live onand will endure for generations to come. Winton Marsalis said it best when he said 'His music sounds like America.' Because of the unmatched artistic heights to which he soared, no one deserved the phrase “beyond category” more than Ellington, for it aptly describes his life as well. He was most certainly one of a kind that maintained a lifestyle with universal appeal which transcended countless boundaries.

Duke Ellington is best remembered for the over 3000 songs that he composed during his lifetime. His best known titles include; 'It Don't Mean a Thing if It Ain't Got That Swing', 'Sophisticated Lady', 'Mood Indigo', “Solitude', 'In a Mellotone', and 'Satin Doll'. The most amazing part about Ellington was the most creative while he was on the road. It was during this time when he wrote his most famous piece, 'Mood Indigo'which brought him world wide fame.

" class="ng-binding ng-scope">Duke Ellington And His Orchestra

© 2010-2020 HIGHRESAUDIO


Tara Maya

Faery Realms-final fix

*Purchased individually, these books cost over $15 – List price $9.99 – Save 90% – Now on sale for only $0.99 cents!*

Download the bundle from Amazon, Barnes and Noble, Kobo, Smashwords, or Google Play.

Best for ages 13 and up.

Faery Realms Final 3D

THE WITHERING PALACE (A Dark Faerie Tale 0.1)ALEXIA PURDY *Exclusive Content*

Untold darkness rules the Unseelie realm of the Land of Faerie. Hidden in this vast area, Aveta, the future queen of the Unseelie Army, perfects her gifts over lifetimes. Learning that magic isn’t the only way to manipulate the world around her, this naive girl grows into a woman of strength and cunning, ultimately becoming one of the most feared leaders in Faerie.

High-tech gaming and ancient magic collide when a computer game opens a gateway to the treacherous Realm of Faerie. Jennet Carter never thought hacking into her dad’s new epic-fantasy sim-game would be so exciting… or dangerous. But behind the interface, dark forces lie in wait, leading her toward a battle that will test her to her limits and cost her more than she ever imagined. (65 pages)

HOOD & FAE (Daughters of Red Riding Hood)TARA MAYA *Exclusive Content*

Harmony’s life will never be the same… Every day is just as normal, and just as boring, as the one before it… And then the Carnival comes to town. Suddenly, Harmony’s small town world is overtaken by the handsome Kieran and she discovers that not all fairy tales are pretend. (140 pages)

“Turmoil, heartache and unexpected romance – all three are immersed into this tale of the Fae.”— Craving YA Reads review of Dark Promise

“Enter the World of the Fae: Magical worlds aren’t just for young adults, I enjoyed this tale and look forward to reading the other books in this series.” –ParaNormal Romance Reviews of The Dark Fae

“Once I started this book I could not put it down, I had to know what happened next.” ~ Amazing Book Come To Life review of Once (Gypsy Fairy Tale)

A stand-alone short science fiction novel.

Publication year: 2013
Format: ebook
Page count: 192 including several appendixes and glossaries

The book is set between 5025 and 5040, covering many years of the main character’s life, spent mostly in warfare.

The story starts on the year 5025 when humankind has spread to many planets and many galaxies. The story is set in a different planet, Neraka, which is pretty hostile place to live. The humans have to wear breathers and full bodysuit, which has been changed into armor, called “kits” in order to stay alive. On this planet, feudal lords control the machinery and therefore the population. They use various mind control technologies on the thralls, who are the lowest class workers on the planets (their minds are wiped to make them compliant), and even on their wives to make them loyal to their husbands. The lords also use combat mechs.

Charlie Cooper is a Fredder. He’s part of a community which is fiercely independent from any feudal lords and refuses to have their minds altered. He’s 18 and about to me married to his sweetheart Bernisse. After they exchange their vows, Charlie leaves for the traditional walk but during it he notices that a heavily armed mechs are advancing on his village. He gives warning but lord Domany tries to blackmail and threaten the village into giving him food and repairs. Charlie is close to his mech and manages to ambush the lord. However, the villagers find out that the lord is badly wounded and give him aid. This is something of a mistake, because another, more powerful lord is after Domany. It turns out that Domany is very much down on his luck: he’s wounded and has only a couple of mechs to help him. However, the Fredders don’t want to ruled by the conquering lord, so they agree to help Domany.

In the middle of battle, Charlie’s new wife Benisse is abducted by the attacking lord’s henchmen. Charlie is wounded and knows that he has no way of rescuing Benisse. But Charlie finds himself in the land of the Sulphines, almost mythological aliens whom nobody has seen in hundreds of years. They are supposed to be sulfur-based and retreated after humans settled on Neraka. However, now their Queen confronts Charlie and gives him a choice: to die or to use the helm of Brin. The helm has the memories of a thousand years of warfare but using it would be a Fredder’s greatest nightmare; losing his mind to the personalities and memories in the helm. Charlie hesitates but he knows it’s the only way to save his wife, even though he will be dead to her.

At the start of the book, Charlie is a pretty simple farm boy. He knows the dangers of the land he lives in and his culture and behaves accordingly. For example, he’s never seen Benisse without her breather until the wedding ceremony. He’s also somewhat reckless when he defends his town but he knows what he can do and takes calculated risks. When he puts on the helmet, he doesn’t feel any difference, but it’s very clear to the reader: his narrative voice changes from a (readable) brogue to more standard English. Later he even gets a British accent. He also becomes more ruthless and struggles to keep his initially good character.

The book has a lot of fighting, especially during the latter half. However, I was more interested in the various cultures. Except for the Fredders, they all use technology to alter memories or personalities. However, they use it somewhat differently. Neraka’s lords use it to keep other people in line but we find out that this a rather old-fashioned way of using the tech. The lords also have duels between single warriors and their armored mech rather than wars, which is very medieval.

The central themes of the book are what is real and what is not, and whom can you trust. When other people can change your memories, your emotions, your thoughts, what can be real?

Today I’m excited to join the blog tour for Blood, the sixth book in Tara Maya’s The Unfinished Song series which is set in a fantastical Neolithic Age. You can buy Blood and all Maya’s other books via Amazon and other book retailers. Tara Maya’s blog has lots of information about writing in general in addition to her book info.

Publication year: 2013
Format: ebook, Kindle
Publisher: Misque Press

The previous book in the series, Wing, ended in a very exciting cliffhanger and I was anxious to continue the series. Blood continues immediately where Wing ended.

Dindi has been captured by Umbral who is a Deathsworn. The Lady Death commanded Umbral to kill Dindi, her love Kavio, and all of the remaining Aelfae. Umbral has already killed Kavio but now he and Dindi face a common enemy, the Bone Whistler who is secretly one of the two Aelfae left in the world. The Bone Whistler has a plan to kill all of the humans and to resurrect the Aelfae who were killed in a war with humans. Dindi and Umbral reached an uneasy truce to kill the Bone Whistler before he can set his plan into motion.

However, a part of it is already working. Six Aelfae have been resurrected, brought forward in time from the past. Also, because of the resurrection magic the other remaining Aelfae, Vessia, has lost her memories of the intervening time. Vessia married a mortal man and had a son with him but now she doesn’t remember that. All she wants is to rid the world of humans because her last memories are of the brutal war between the Aelfae and the humans.

Meanwhile, Finnandro the champion of the Green Lady had been forced into uneasy truce, as well. His tribe and another fought each other but now Finnandro is forced to work with the enemy tribe in order to kill Umbral. Unfortunately, the leader of the enemy tribe, the Orange Canyon, is no other the Bone Whistler himself and he manages to charm Finnandro into a willing ally.

However, the resurrected Aelfae don’t know that there’s a traitor among them. Their greatest enemy, the Lady Death herself, has disguised herself as one of them and plans to kill them all.

Blood is just as good as rest of the series. The plot has twists and turns which I, at least, couldn’t see coming. We get to know a lot more about both Umbral and Finnandro in flashbacks about their past.

Deception seem to be this book’s theme. A lot of people are deceived in this book. Not only has the Lady Death disguised herself and deceiving the Aelfae but the Bone Whistler uses his powers to charm other people to trust him, while he’s advancing his plan. He’s also an accomplished liar and can turn any event to his advantage when given enough time to either explain or let other people draw their own, wrong, conclusions. Vessia has lost her memory and is convinced that she couldn’t possibly have married a human. Umbral has the Obsidian mask which can make anyone who looks at him see the person they want to see.

I continue to be fascinated with this Stone Age world. Most of the tribes don’t have a king or nobility but are instead lead by elected leaders. The one exception is of course the Bone Whistler who uses his charm and power to keep his people in line. The Orange canyon tribe has the Raptor Riders and their slaves who can change their shape into huge raptor birds. The Aelfae are a bit puzzled by this and claim that the Riders have also shape changing magic. The tribe is pretty cruel towards their own members, too.

The new element in the book are the resurrected Aelfae. We get to see how they lived during the war and presumably before it. They seem to be pretty self-centered, tormenting Dindi for their own amusement. They’re also dismissive of humans because they see the humans as enemies. When the Bone Whistler revealed to them that he lives among humans as their war chief, the Aelfae were shocked and dismayed. However, they agree to live as guests in the human village and even take part in their celebrations.

A word of warning, though: the book has pretty gruesome torture scenes. The torture is seen both from the eyes of the tortured and the torturer, and it’s quite disturbing.

Today I’m excited to join the blog tour for Wing, the fifth book in Tara Maya’s fantasy series Unfinished Song which is set in a fantastical Neolithic Age. You can buy Wing and all Maya’s other books via Amazon and other book retailers. Tara Maya blog‘s has lots of information about writing in general in addition to her book info.

Publication year: 2012
Format: ebook
Page count: 198 in pdf
Publisher: Misque Press

The Unfinished Song series has been full of surprising twists and the latest book continues in the pattern. Maya isn’t afraid to do nasty things to her characters, even killing them. The book starts immediately after the shocking ending of the previous book, Root.

Most of the point-of-view characters are the same as in the previous book. Dindi is the main character of the series but the other POV characters were introduced in Root or Sacrifice.

The young maiden Dindi has just been kidnapped by one of the Deathsworn, Umbral. Umbral is determined to kill her because she has a connection to the high faeries, the Aelfae, and Umbral is convinced that Dindi can help the Aelfae return to the world. The Deathsworn have killed almost all of the Aelfae and want to now finish their job. However, Dindi is a resourceful girl. When Umbral tries to kill her almost immediately, Dindi manages to trigger a vision which delays Umbral and makes him think that Dindi would be useful to him. So, Umbral suggests a bargain: as long as Dindi will see visions, Umbral doesn’t kill her.

Unfortunately, Umbral thinks that Dindi’s visions are about the current day and about the White Lady, Vessia, who is one of the few remaining Aelfae. While Dindi’s visions are about Vessia, they are twenty years in the past. Dindi is trying to be as vague as possible about her visions, so that Umbral doesn’t realize it. Also, Umbral has confessed that he has killed Dindi’s beloved Kavio. Because Umbral has one of the six objects of power, he looks like Kavio to Dindi. Dindi is looking for a way to avenge her beloved but the way Umbral looks like Kavio is unsettling to her.

Umbral is a devotee of the Black Lady, Death. He has done terrible things in the past but doesn’t remember anything from the time before he became a Deathsworn. The Deathsworn, the servant of the dead, separate the dead and the dying, and most of the time they kill people who are too badly wounded or diseased to live. However, Umbral has also attacked, tortured, and killed healthy people which Deathsworn shouldn’t do. He leads the Deathsworn group with intimidation, violence, and fear.

However, Umbral is becoming increasingly attracted to Dindi. He kills people either with weapons or by absorbing the magical Chromas, the life force, of his victim. Sometimes when he has absorbed someone, the victim’s memories still linger with him. Umbral’s immediate goal is to kill the White Lady because he’s convinced that the faeries will kill humans if they aren’t killed first. He seems relieved when he has an excuse to keep Dindi alive. They must track the White Lady and work together while traveling through wilderness.

Vessia, the White Lady, has been kidnapped by her husband’s brother and niece. They loath Vessia and are trying to use her as a leverage against her politically powerful (human) husband. However, even though Vessia has lost her immortality, she’s far from helpless.

Finnadro the Wolf Hunter is the Henchman of the Green Lady, Vessia’s sister faery. He loves his mistress unreservedly and has even rejected human lovers in favor of her. He’s a great hunter and warrior. In the past, Umbral tortured him and left him for dead. Now, he has an excuse to hunt Umbral and maybe save both Dindi and Vessia.

In the previous book Dindi and her traveling group of dancers were staying in a village called Green Woods. The village was attacked by a group of Raptor Riders and the warriors of the Orange Canyon tribe. The fight was brutal and many died on both sides. The Green Woods warriors want revenge and send a group to the Orange Canyon lands. Among the Green Woods warriors are a few of Dindi’s friends.

Tamio and Hadi are both young warriors and Dindi’s friends. However, while Tamio is expecting the next battle and the glory he will get from killing other warriors, Hadi knows that he’s a lousy warrior (and a hunter) and he’s dreading the fight, thinking that he will be killed. Also, Hadi wants to protect his sister who was married just before the fight – and widowed during it. One of the shape shifters rescued Hadi during the fight and now Hadi owes a liftdebt to him. However, the shape shifter has his eye on Hadi’ sister and takes her as a servant in Hadi’s place. Hadi isn’t happy about it but can’t change it.

Generally, I don’t really care for books with multiple POV characters because most of the time it’s hard to distinguish the young white male warriors from each other and sometimes they aren’t needed to advance the plot. However, in this series all of the POV characters have been so different from each other that I have no problem distinguishing them from each other, even when they travel in the same group. Also, they all have their own problems which don’t necessarily directly tie into the main plot. Even Tamio has an interesting, and appropriate, subplot. I could barely stand him in Sacrifice where he was first introduced.

Most of the book is written in third person but each seven chapters start with a short section in first person. They all give great insight to the character because they all describe a significant event in that person’s history. For example, Finnadro’s first person section is his memory of how he won himself the Shining name of Wolf Hunter and we also get to know a little about his family and tribe. We also get insight into Dindi (in fact she gets insight into her own feelings) but her narration is for the present and not from past.

I’m just wondering a little about the ending. Dindi developed very handy new abilities earlier in the book. Why didn’t she use them at the end where they would have been very, very useful? I suspect it’s because they are new and she couldn’t rely on them but she could have even thought about them or tried to use them.

Each book in the series so far has expanded the world and this book is no exception. We get to know more about the Raptor Riders. They are humans who have enslaved magic users who can change their shape into various raptors. However, the riders have their secrets, too. In Root we found out that some of the Green Woods people are born as shapeshifters, wolves. Most of them are banished into the wilderness and are called wildlings. Finnadro knows them and can even ask them for help. Apparently, it’s also his job to make sure that they don’t bother the tribesmen. Now, we see that some of the shapeshifters can shift into different forms but they seem to be even more shunned than the wolves. We also get to hear a couple of the old legends which I found fascinating. We also get to see a few scenes from the war between humans and the Aelfae. In Dindi’s visions about Vessia we get to see how a human and a fae try to build a life together and it isn’t easy. And Dindi and Umbral encounter a new human clan during their travels.

Wing is another great continuation with twists and turns. It’s ends with a great cliffhanger!

The fourth book in the fantasy series Unfinished Song which is set in a fantastical Stone Age.

Publication year: 2011
Format: ebook, pdf
Page count: 150
Publisher: Misque Press

Some time seems to have gone by after then end of the previous book, Sacrifice. The group of Initiate Tavaedis, magical dancers, have been formed into a performance troop under their teacher Abiono and they have traveled away from the Yellow Bear clan. Unfortunately, Abiono is an old man and unable to control the young dancers. Dindi is working as the Tavaedi’s serving maiden and has been given permission to learn the dances openly. However, a spoiled girl from the previous book, Kemla, is part of the group and as the most popular girl is apparently leading the women of the group. She seems to hate Dindi because the red fae, whom only Kemla and Dindi can see, constantly tease Kemla that Dindi is a better dancer than Kemla and that Dindi should take Kemla’s place as the star performer. Since Kemla can’t punish the fae or make them silent, she takes her rage out on poor Dindi, treating her as a slave. She takes things so far that she urges Tamio to sleep with Dindi so that Kemla can reveal it publicly and Dindi will be thrown out of the party. Apparently, there are no consequences for Tamio?

Now Dindi she can practice as much as she wants even though the others are pretty cruel to her (and once again the adults just stand by and let this happen). However, she was forced into a bargain with the fae in the previous book; she will have to find a way to lift the Curse from the Aelfae and bring them back from the dead. Dindi is using her corncob doll to find a way to do that. However, Tamio is hounding her so there’s not much time to do it. She also found out that there’s a hex on her family and she’d like to find a way to undo that, as well.

Meanwhile Kavio’s fae mother Vessia, known as the fae White Lady, is determined to help her son by finding women who could become the next Vaedi, the wife of the war chief. However, she’s being held captive by her own nephew. She manages to escape with the help of a young warrior and they set out to flee Vessia’s own tribe. They come across the Lost Swan tribe looking like a pair of beggars. Kemla denies them hospitality but Dindi shares her meager food and shelter with them.

Then, a group of warriors attack. They ride on big birds and kill some of the people before the nearby tribes come on horseback to aid them.

Umbral is a new characters and so is his group. He is a leader of a group of Deathsworn; those who serve the Lady Death. They all seem to have some sort of physical deformity and so they have been sent to the Deathsworn. Even the women have physical deformities and this was a refreshing change form the “flawless skin” princesses of fantasy. They are eager to kill people and are investigating a magical plague which devours people’s spirits. They also want to kill Vessia and the next Vaedi. Umbral comes across traces of a magic both ancient and fresh. Quickly, he becomes obsessed with the maiden who has left such traces behind her.

Dindi’s visions with the Corn Maiden seem to be over and she now sees into the life of Mayara, who is the last survivor of an Aelfae settlement. As a little girl, Mayara suffers horrible things: her mother cuts off her wings so that the humans wouldn’t kill her. Then her mother hides her just before the humans come and little Mayara sees her people slaughtered. Later, she wanders alone in the woods until a human finds her and takes her home, to live among his family.

Even though the group is performing and working magic in the Lost Swan tribe, Dindi’s tribe, she’s treated poorly. She is sent to sleep in a very cold hut and given only a small amount of food. However, this doesn’t seem to be her home tribe because we don’t see her parents or other close family.

There are some differences in the setting. Specifically, riding mounts. The Raptor Riders have large predatory birds which they use to ride on and the Broken Basket and Full Basket clans have horses. Unfortunately, the inclusion of horse made the setting less unique to me. Part of my enjoyment of the books have been their setting which is rather different from all the pseudo-medieval settings which are very common in fantasy. The use of horses makes the setting more familiar and less unique. In the previous book, there was a mention of gold and jewelry so the people have the means to smelt metals and work them quite intricately.

Two of the point-of-view characters are unfortunately pretty insufferable to me: Kemla and Tamio. They both start as arrogant and self absorbed to the point that they have no compassion or empathy to anyone else. We get to know more about them but they never really reform. Tamio is a unabashed womanizer and the best thing that can be said about him is that he isn’t a rapist. The Deathsworn are interesting and I hope we get see more of them.

I also found it a bit weird that while there are lots of talk about sexual conquests, none of the women worry about getting pregnant. However, in the previous book there was a brief mention that illegitimate kids are “unwelcome” and that a man will have to either marry a girl he gets pregnant or pay with foods and other stuff. The latter seems to be more common. Yet, the woman is expected to care for the kid and surely a poor woman without much kin, such as Dindi, or an ambitious Tavaedi such as Kemla, should be worried about being able to rise a kid. Of course, dwelling over such things would slow down the pace and possibly bore the reader. Unfortunately, this society too has the sexual double standard for women and men.

There are two new tribes in the book, the Green Woods tribe and Raptor Riders. Both are warrior clans which have quite different customs than the tribes we’ve seen so far. For example, both men and women can be warriors in both clans. They also have shape shifters. The Raptor Riders use huge birds and enslave them while some of the Green Woods people can turn into wolves. However, these wolflings aren’t tolerated until they can control themselves. They are banished into the woods to presumably learn control but, not surprisingly, most seem to live in the woods all their lives, and attack people occasionally. A bit disappointingly, at least in the Green Woods tribe the women have to tend to their chores in addition to being warriors while the men sit and talk.

The plot is again fast paced and full of twists, some of them unexpected. The book ends in a cliffhanger. More things happen in every short book than some established writers manage to put into 600 page books.

The third book in the fantasy series Unfinished Song which is set in a fantastical Stone Age.

Publication year: 2011
Format: ebook, pdf
Page count: 173
Publisher: Misque Press

The book starts soon after the ending of the previous book, Taboo.

The peace party is returning back to the Yellow Bear tribe, defeated. Kavio was the leader and the organizer of the party but he was betrayed almost at every turn. High ranking warrior and an obnoxious bully Vultho, who had been a part of the peace party, had ordered his warriors to attack an enemy clan hold and so in effect ended any chance for peace. The rest of party had been captured and tortured but managed to escape. Kavio has been told that if the peace party isn’t a success, he will die.

When they are near Yellow Bear clan hold, they are warned that Vultho is the new War Chief. Kavio is able to bluff his and his party’s way back into the tribe. However, Vultho doesn’t make his life easy. The clan prepares for war.

Meanwhile Dindi, the clumsy and magickless maiden, has her own troubles. She is still determined to learn the magical dances and Kavio is willing to teach her, in secret. However, this means that Dindi spends pretty much all of her time away from other people and her age mates resent that. Dindi has a hexed corn cob doll which sends her visions, sometimes in very unfortunate times. She’s trying to destroy it but hasn’t succeeded.

Dindi’s friend Gwenika also isn’t very popular and she finds out that the young Initiates have a cruel tradition: they choose from among them one hapless person, who is called the Duck and torment him or her until the Duck kills himself or herself. To her horror, Gwenika finds out that she is one of the candidates this year. However, her sister Gwena has missed Gwenika and stands up for her. Unfortunately, Dindi is chosen as the Duck.

Brena, the adult magic dancer, gets back to teaching the Initiates. But she can’t stop worrying about the coming war and she doesn’t have any patience for the youngsters anymore. When she confesses this, she’s sent to the other tribes as an envoy to gather allies against the Blue Waters tribe.

Even though Kavio is quite young, he has already a lot of enemies. Yellow Bear’s former War Chief Hertio is an old friend but even Hertio is trying to use Kavio to his own advantage. Kavio has to keep alert against any ploy the current War Chief would use against him and train a group of warriors in secret, because Vultho would never allow him to train them.

This third book has more romance than the previous books. Brena and Rthan became lovers in the previous book, Taboo, despite the fact that they come from warring tribes and that Rthan was Brena’s slave. At the end of the book, Rthan decided to stay (and fight) with his own tribe and they separated. Now, they are set directly against each other. Brena is the champion of a fae known as the Golden Lady. The Golden Lady is dying from a wound from the Black Arrow. Only if the Black Arrow takes another’s life, can the Golden Lady be healed. The fae hinted that there are only a few people who can be this victim but Brena is looking for a way to kill someone else than people whom she considers innocents. Rthan is a warrior of the Blue Waters tribe but he’s also a champion of the fae Blue Lady who wants to kill the Golden Lady because they are ancient enemies. The Blue Lady has taken on the form of Rthan’s eight year old daughter who was killed by the Yellow Bear tribe’s current war leader. In addition, the chiefs of their respective tribes have sent Rhtan and Brena to other tribes as envoys to try to persuade the other tribes to join them.

Dindi and Kavio are another romantic couple with romance troupes. They both struggle against their attraction to each other, thinking that the other can’t be attracted to her/him or even if he/she is, they can’t be together because Dindi doesn’t have any magic and Kavio is a powerful magic user. Still, Kavio is teaching Dindi the forbidden magical dances and they have to spend a lot of time together. Fortunately, they don’t dwell on their feelings too much.

There’s a new POV character, Tamio. However, he’s seen only a few times when the plot demands it. He’s a young Yellow Bear Initiate, ambitious and not too picky about whom he serves. He’s in the group Kavio is training to fight in water but later he doesn’t mind siding with Vultho when it seems advantageous. He’s also handsome and uses that to get girls.

Unfortunately, I found Vultho to be almost a caricature as a villain. He’s crude, arrogant, and quick to anger. He has celebrations which wastes valuable food. He also doesn’t seem to be able to plan much. This is somewhat explained with his background; apparently he used to be bullied a lot and when he now has power, he uses it to his own satisfaction.

However, I can believe that the teenagers would choose one hapless person amongst themselves and hound him or her to death. That’s just the sort of cruelty and malice teens are capable of and don’t necessarily even think much about. It’s said that all Initiates know about this, which means that all of the adults know about it, too, and yet nobody interfered. Tacitly, all the adults seems to think that this is just ok which makes them pretty awful.

The rest of the cast are more complex. Gwenika turning on her (former) friend is something that some people do when there’s a right reason. Dindi struggles to fulfill her life long dream against all the odds and Kavio tries to keep the peace.

The faeries have a much larger role in Sacrifice than in either of the previous books. They aren’t just manipulating humans but fight themselves. We also get to meet all six types of High Fae and get to know a little about their relations, and about their physical forms. The faeries have six types, the same as there are six magical colors.

The plot is again very fast paced and full of twists. Several of the plot lines and questions from the previous books are answered here but a couple of new plot lines are introduced with high stakes. This was a great and surprising continuation to the series, and I have a feeling that some of the cast is going to change.

Publication year: 2011
Page count: Kindle doesn’t have page numbers
Size: 451 KB
Format: ebook

Next Page »


A Level Playing Field

Everyone is treated fairly.

Once a candidate has completed an application, five Evaluation Panel members will be assigned to score each application that is determined valid or eligible during our Administrative Review process. Evaluation Panel members will offer both scores and comments for each of four distinct traits in our rubric. Each of the four traits will be scored on a 0-5 point scale, in increments of 0.1. Those scores will combine to produce a total score. Examples of possible scores for a trait are… 0.4, 3.7, 5.0, etc.

The most straightforward way to ensure that every candidate is treated by the same set of standards would be to have the same Evaluation Panel members score every valid application; unfortunately, due to the number of candidates that we will receive, that is not possible.

Since the same candidates will not be scored by every Evaluation Panel member, the question of fairness needs to be carefully explained. One Evaluation Panel member scoring a candidate may have a more critical disposition, giving every assigned candidate a range of scores only between 1.0 and 2.0, as an example; meanwhile, another Evaluation Panel member may be more generous and score every candidate between 4.0 and 5.0.

For illustrative purposes, let’s look at the scores from two hypothetical judges:

Judge1scorestop Judge2scorestop

The first judge is far more generous in scoring than the second one, who gives much lower scores. If a candidate’s application was rated by the first judge, it would earn a much higher total score than if it was assigned to the second judge.

We have a way to address this issue. We ensure that no matter which judges are assigned, each valid application will be treated fairly. To do this, we utilize a mathematical technique relying on two measures of distribution, the mean and the standard deviation.

The mean takes all the scores assigned by a judge, adds them up, and divides them by the number of scores assigned, giving an average score.

Formally, we denote the mean like this:

The standard deviation measures the “spread” of a judge’s scores. As an example, imagine that two judges both give the same mean (average) score, but one gives many zeros and fives, while the other gives more ones and fours. It wouldn't be fair, if we didn’t consider this difference.

Formally, we denote the standard deviation like this:

To ensure that the judging process is fair, we rescale all the scores to match the judging population. In order to do this, we measure the mean and the standard deviation of all scores across all judges. Then, we change the mean score and the standard deviation of each judge to match.

We rescale the standard deviation like this:

Then, we rescale mean like this:

Basically, we are finding the difference between both distributions for a single judge and those for all of the judges combined, then adjusting each score so that no one is treated unfairly according to which judges they are assigned.

If we apply this rescaling process to the same two judges in the example above, we can see the outcome of the final resolved scores. They appear more similar, because they are now aligned with typical distributions across the total judging population.

Judge1scaledscoresbot Judge2scaledscoresbot

We are pleased to answer any questions you have about the scoring process. Please feel free to ask any questions on the discussion board. Please REGISTER today to begin developing your application.

Want to be recognized for your work in clean energy?

Only 26 days to register.

Launched at the Paris Climate Conference in 2015, Mission Innovation (MI) is a global initiative of 24 countries and the European Commission, on behalf of the European Union, seeking to dramatically accelerate global clean energy innovation and make clean energy widely affordable and reliable.

MI countries represent 60% of the world’s population, 70% of GDP, and 80% of government investment in clean energy research. MI countries are taking action to double their public investments in clean energy R&D over five years while encouraging collaboration among partner nations, sharing information, and coordinating with businesses and investors.


Our Mission

We’re shifting campus culture to one of mutual accountability and respect by empowering students to make decisions that contribute to a successful college experience.

Not Anymore by Safe Colleges provides the curricular foundation to educate students about drugs, alcohol, and sexual violence. The goal is to enable stimulating, honest conversations around campus that lead to action, changes in behavior, and a safer atmosphere for all.

We understand that we cannot eliminate alcohol from campus. We can, however, provide students with the tools to engage with alcohol in a safer, more intentional manner. Not Anymore encourages students at USF and around the country to live with purpose and mindfulness, so they succeed in the classroom and in their day-to-day lives.


Cons globalization essay against globalization

We will divide our story in three main parts. In first part we will read What is Globalization? Globalization is the process of international relationships among the peoples of different countries, among the companies and the governments and their institutes etc. The aim of this process is to become united on international platform in the field of international trade, environment, culture, health, economic developments and many other field according to the needs of all countries.

Cons globalization essay against globalization

Pros and Cons of Globalization Globalization refers to the aspect of international integration. The opening of agencies by governments that open up chances for their local citizens to interact with members and citizens from yonder continents is the backbone of globalization.

However, the originality of globalization is founded on technological innovations and advancement. The one major talking point that has evolved and stood around globalization is the issue of employment. In some aspects, employment has been amplified by globalization to an incredible extent, scaling to heights that no one ever thought would be achieved.

In this essay about globalization and employment, we will see how the integration of the whole world into a synchronous system has impacted employment across the world.

Cons globalization essay against globalization

The Concept of Globalization and Employment Globalization has played a central role in the business arena, more specifically targeting the distribution of workforce. Born by early merchants and propagated diversely over the vicissitudes of time, employment has developed new edges and facets thanks to globalization.

The history of it all can be traced back to the industrial revolution, where improvement of technology and infrastructure enabled people to start working for massive production.

Over time, people traveled from different places in pursuit of employment and landed in these industries and plantations. The trend grew over time, and before long, people were traversing continents and oceans for the same.

To date, there are numerous examples of employment that has its roots in globalization. In this essay on globalization, we choose to focus on the development of jobs owing to globalization and the negative impacts of this phenomenon as well. Globalization Pros and Cons Essay Globalization Has Had a Positive Outcome Regarding Employment As with every other arising paradigm that affects the usual procession of the daily running of lives on earth, globalization has brought with it many merits and demerits as well.

The creation of employment is an outstanding feature as viewed from the perspective of globalization. In other words, technological innovation is the outcome of a worker somewhere who has been at it for a while getting themselves out of the bed to make a living.

When sportspeople are contracted to compete for foreign countries, that aspect is in itself under globalization as well. Multinational companies provide the best example to highlight in a globalization essay.

They are companies that expand their operations beyond the territorial borders of their mother country. They venture into new grounds and establish themselves there. In doing this, they employ the locals and the natives in the satellite companies.

This is one of the benefits of globalization. Other examples of employment that has resulted from globalization include: Sourcing of foreign experts to assist in local ventures such as training and education Invoking foreign investors to establish businesses in a country Getting experts from developed countries to share and install infrastructure in developing countries A team of employees from different countries working on a contract with an agency Trade deals between governments that see labor force exchange between the concerned countries The list can be extrapolated as the examples are many.

All these are situations that have risen up and brought about employment for people, and they all came up as a result of globalization.

The Demerits of Globalization on Employment As with the job creation, a globalization pros and cons essay should deal with the observation of the harsh face of the concept in some situations. In as much as globalization is an excellent idea, it has brought about agony and cost some people in different countries their rightful place in their employment profile.

People who ought to have standard and worthy jobs end up doing odd jobs or lacking them altogether because of the effects of globalization. The problems develop primarily out of the competition.

For instance, a multinational company that sprawls over to foreign countries will, for most of the time, be focusing on the local natives to provide the primary forms of labor. The initial employees and those who feel they had the right to get employed by the company over the new employees will then be dejected.

Competition for the limited slots of employment also comes into play, and the end result is always cheaper labor for the employer but substantial wages for the employee.

Pros and Cons of Globalization: Controversy and Discussion

Exchange programs as discussed in many globalization essays by some writers have proved to be detrimental to the job security of citizens, especially in the developing countries.

The importation of expert workers into a nation puts the locals at risk of unemployment. This is always definite. Many globalization pros and cons essays have been written to complement similar books and agitate for fairness in such a system. It is difficult to manage a society where the owners of the community itself are disgruntled, especially by matters relating to economy and poverty.

How the Situation Can Be Settled The government has the obligation to serve and protect the citizenry, not just from physical attacks but also from the artificial economic crisis. A globalization essay example will highlight the fact that when foreigners are prioritized in the job industry over the citizens, a financial crisis is always looming.

This is a precarious situation for a nation. In order to avoid it, the government can regulate the numbers of expatriates or check the employment status of the population so that it strikes a balance between the two.In this globalization pros and cons essay related to businesses, we have reviewed the effect globalization has had on markets, the economy, and the environment.

It is difficult to make a ruling on whether globalization is the best way for the world to go owing to such controversial outcomes and reactions from different zones.

The Pros And Cons Of Globalization

For instance, the. The Pros and Cons of Globalization Essay Words | 4 Pages. It is debated whether globalization is an advantage or disadvantage to our society.

Cons globalization essay against globalization

What are the Pros and Cons of Globalization Essay & Examples shoaibraza August 1, Global 2 Comments 31, Views Most of students wants to know about globalization, this essay and examples will complete you concept about the pros and cons of globalization.

In most globalization pros and cons essays, writers forget that employment is as meaningful as any other aspect of human livelihood. Globalization Has Brought About Cultural Erosion The rise of civilization can also be viewed as the downfall of cultural heritage.

For better or worse, globalization is a complex topic, and if you’re writing a pros and cons essay about it, you’ll want to make sure you find good sources to back up your ideas. In this blog post, I’ll provide 20 globalization articles to help you get started.

Globalization pros and cons: economic, cultural, political - netivist


I was diving into Symfony framework (version 4) code and found this peace of code:

$env = $_SERVER['APP_ENV'] ?? 'dev';

I'm not pretty sure what this actually does but I imagine that it expands to something like:

$env = $_SERVER['APP_ENV'] != null ? $_SERVER['APP_ENV'] : 'dev';

Or maybe:

$env = isset($_SERVER['APP_ENV']) ? $_SERVER['APP_ENV'] : 'dev';

Someone has any precision about the subject?

To all the people who answered the question: thank you To all the people who marked my question as negative because there's already a similar question (PHP ternary operator vs null coalescing operator):

It is true that both questions are very similar. However it is hard for everybody to imagine that the "??" is called the coalescing operator.

Otherwise I could easy find it on the official documentation:

However, for someone who didn't know that this feature was added in php 7 it's more likely to type:

"php ?? operator" or "php double question mark operator"

And here is why my question has an added value.

I ask you to, please, reconsider your negative feedback. Thanks

Regards, Epixilog

  • 8
    @gp_sflover for "php 7 operators" ok, you have chances to find it. With "php operator"...good luck :) And that's what anyone who never saw it before would do In my opinion it's not completely intuitive to think that this is a new feature that was only included in the latest version of php. Anyway, maybe I'm wrong and I just have to sleep and wake up to realize it. Thanks for your time :) – elkolotfi Dec 4 '18 at 14:45
  • 6
    I'm marking this for re-open. It is not a simple matter of typing in 'php ??' or anything similar in the search box to find the answer. Even now, with this question 'php' and '??' will not display this question. I recommend editing the title to include '??' so it is easier for others to find. Second, @epixilog, marking a question as duplicate is in no way a reflection on you. Even if it's closed, it worked as intended as you found your answer. It's marked that way to ensure we keep all the information together, not to judge people asking questions. – Andrew T Finnell Dec 4 '18 at 19:50
  • 3
    @AndrewTFinnell It's not duplicate mention that bothers me actually it's the -5 feedback I got for my question ;( – elkolotfi Dec 5 '18 at 8:22
  • 9
    I upvoted because I also googled for „PHP double questionmark“. This is the first result and answers perfectly. – Zim84 Mar 20 '19 at 19:42
  • 3
    Same here exactly, thanks to this question I just found within less than 10 seconds what ?? means in PHP by literally typing "php double question mark operator" and confirming what I thought it would be. Excellent – kon Mar 21 '19 at 12:44

It's the "null coalescing operator", added in php 7.0. The definition of how it works is:

It returns its first operand if it exists and is not NULL; otherwise it returns its second operand.

So it's actually just isset() in a handy operator.

Those two are equivalent1:

$foo = $bar ?? 'something';
$foo = isset($bar) ? $bar : 'something';

Documentation: http://php.net/manual/en/language.operators.comparison.php#language.operators.comparison.coalesce

In the list of new PHP7 features: http://php.net/manual/en/migration70.new-features.php#migration70.new-features.null-coalesce-op

And original RFC https://wiki.php.net/rfc/isset_ternary

EDIT: As this answer gets a lot of views, little clarification:

1There is a difference: In case of ??, the first expression is evaluated only once, as opposed to ? :, where the expression is first evaluated in the condition section, then the second time in the "answer" section.

  • $foo = isset($y++) ? $y++ : 'something'; This doesn't work. Can not use isset on the result of an expression. – ascsoftw Sep 4 '19 at 12:45
  • @ascsoftw sorry, removed until i find of better example – michalhosna Sep 5 '19 at 20:25
  • Regarding the last edit : the number of times it gets evaluated also depends on the internal implementation of the ?? operator (is it really once?) – Tomek Jan 10 at 10:37
$myVar = $someVar ?? 42;

Is equivalent to :

$myVar = isset($someVar) ? $someVar : 42;

For constants, the behaviour is the same when using a constant that already exists :

define("FOO", "bar");
define("BAR", null);

$MyVar = FOO ?? "42";
$MyVar2 = BAR ?? "42";

echo $MyVar . PHP_EOL;  // bar
echo $MyVar2 . PHP_EOL; // 42

However, for constants that don't exist, this is different :

$MyVar3 = IDONTEXIST ?? "42"; // Raises a warning
echo $MyVar3 . PHP_EOL;       // IDONTEXIST

Warning: Use of undefined constant IDONTEXIST - assumed 'IDONTEXIST' (this will throw an Error in a future version of PHP)

Php will convert the non-existing constant to a string.

You can use constant("ConstantName") that returns the value of the constant or null if the constant doesn't exist, but it will still raise a warning. You can prepended the function with the error control operator @ to ignore the warning message :

$myVar = @constant("IDONTEXIST") ?? "42"; // No warning displayed anymore
echo $myVar . PHP_EOL; // 42
  • 1
    isset already returns false for null values – Timo Huovinen Mar 22 '19 at 15:26
  • 1
    @TimoHuovinen yep, edited – Cid Mar 22 '19 at 15:38
$x = $y ?? 'dev'

is short hand for x = y if y is set, otherwise x = 'dev'

There is also

$x = $y =="SOMETHING" ? 10 : 20

meaning if y equals 'SOMETHING' then x = 10, otherwise x = 20


Why Gregg Shorthand Is Still Helpful Today

Like many other skills, pen stenography today...

Like many other skills, pen stenography today has been largely superseded by mechanical and digital technological advances. For many, this includes the twentieth century’s most popular and efficient shorthand system. Gregg Shorthand, invented by John Robert Gregg in the late nineteenth century, is considered a valuable tool by the waning number of reporters still using it.

The outstanding efficiency of Gregg Shorthand derives from its basic principles, all of which are aimed at streamlining the writing and reading process as much as possible. This focus has resulted in creating a shorthand system that is uniquely adapted for high-speed note taking.

gregg-shorthandWriting Mechanics Are Optimized for Speed

One of the unique features of the Gregg system is the degree to which it has analyzed the mechanics of handwriting and pared them down to the bone. Ordinary longhand letter forms are simplified to a basic array of four shapes. Shallow curves or straight lines denote consonants; vowels are written with small hooks or loops. All letters lean to the right, following the way most people would write in longform.

This shorthand method also eliminates ligatures between letters, saving additional time. In fact, the Gregg system incorporates a technique called blending, where pairs of letters that often occur together are blended into one character, again cutting down on the number of strokes.

Phonetic System

Its phonetic base gives Gregg shorthand another speed boost, deleting phonetically superfluous or redundant letters from its alphabet. “C” and “Q” are not used, since the sounds they produce are signified by “K” and “S”. Phonemes that use a combination of several letters, such as “th,” are represented by a single character. Silent letters such as the “gh” in “thorough” are not taken down at all; nor are unstressed vowels.

System of Abbreviations Is Easy to Learn and Read

The greatest time-saver in the Gregg system is its use of abbreviations for some of the most commonly used words. A single character can be used to denote several words at a time. An impressive example is the word “bring:” consisting of five letters in longform, in Gregg shorthand it is shortened to “b-r,” which has its own combination symbol. Thus, the whole word can be taken down instantly with a quick stroke of the pen. This convention is known in the Gregg terminology as a brief form. When mastered and used efficiently, it can amp up writing speeds to over 200 words per minute.

Another time-saving feature is the use of a set of standard abbreviations for common prefixes and suffixes, consisting of a single designated letter that can also vary its meaning depending on its position relative to the word. This can be especially valuable for court reporters, who often have to take down lengthy legal, medical and scientific terms.

Phrasing is another technique that promotes speed. Frequently used blocks of language, or phrases, are combined into highly simplified forms. Typically, each word in the phrase is represented by a letter; the letters are then joined into a continuous form for maximum writing speed.

Intuitive Mechanism

Gregg shorthand achieves its extreme efficiency both by eliminating superfluous writing and by creating a streamlined alphabet where each letter can have multiple meanings that are disambiguated by the letter’s relative position. It is easy to learn because the system of abbreviation is intuitive and based on the way that people naturally process written and spoken language, rather than being a list of arbitrarily assigned codes. The phonetic base allows the reporter to forget about spelling and focus on transcribing sound. The fact that Gregg works with, rather than against, normal human language processing, facilitates both learning the shorthand system and the ease with which shorthand notes can be read.

In addition to streamlining the mental processes of shorthand note taking, the Gregg system is also geared to optimize the physical mechanics of handwriting. It eliminates closed loops and other redundancies, reducing letter shapes to basic components that can be formed at high writing speeds.

How Today’s Court Reporters Can Benefit From Gregg

The work of a court reporter includes transcription of many types of speech, by different people who speak at variable speeds. Some of them may have accents or speech impairment, making them harder to understand. Many will use lengthy professional terms. At the same time, transcription must be speedy and highly accurate.

While many reporters today prefer to use mechanical means of transcription in conjunction with specialized software, the few who use Gregg shorthand do it for good reason. The system is optimized to combine a high degree of accuracy with lightning-fast writing speed. Familiarity with Gregg shorthand can be helpful even for reporters who typically do not use it, as over-reliance on technology can otherwise lead to a lack of skill and understanding. Gregg can also be used with digital handwriting recognition programs to assure both data storage and high speeds.

This blog post is intended only to provide information on Gregg Shorthand. Cook and Wiley court reporters are not able to translate any shorthand documents at this time. Thank you!

411 E. Franklin Street
Suite 205
Richmond, VA 23219
(804) 359-1984

Emergency Assistance 24/7 | 888.433.3767 | Copyright 2019 Planet Depos, LLC | All Rights Reserved

Send this to a friend