text:
### A Pluto.jl notebook ###
# v0.18.2

using Markdown
using InteractiveUtils

# ╔═╡ 280b6629-e391-459d-99fb-26a1b483073b
begin
	using Downloads
	using ARFFFiles
	using ScientificTypes
	using DataFrames
	import StatsBase
	using MLJ
	using MLJDecisionTreeInterface
	using StatsPlots
	using PlutoUI
end

# ╔═╡ 42a4863a-7955-11ec-117a-dbb08de9f921
html"""

Machine Learning using Julia and its ecosystem

Using a Decision Tree Classificator (Part II)

&copy  Dr. Roland Schätzle

"""

# ╔═╡ 2129f49d-9e19-4fa1-b878-22b3c8b66a90
PlutoUI.TableOfContents(title = "Using a Decision Tree")

# ╔═╡ 4ed56657-4dab-4fe6-881f-12d1ae32a83a
md"""
## Setting up the environment
"""

# ╔═╡ 1a86389d-cae3-4848-bef1-b0ef1fe1ba7a
md"""
First we have to specify the needed packages and then we load our 'glass' dataset into a `DataFrame` (see Part I).
"""

# ╔═╡ 377ae933-4d00-40cf-9355-332065094f60
md"""
!!! note

    `MLJ` as well as `StatsBase` export a `predict` function. In order to avoid a collision, `StatsBase` is merely `import`ed (so it's identifiers can only be used by a qualified call like `StatsBase.XXX`).

"""

# ╔═╡ 031cf26d-4fde-4a8a-a7d2-f49200c2fab7
glass_file = Downloads.download("http://storm.cis.fordham.edu/~gweiss/data-mining/weka-data/glass.arff");

# ╔═╡ c5e832ea-4a18-430e-8d0f-305da07e3711
glass = ARFFFiles.load(DataFrame, glass_file);

# ╔═╡ 116d483d-6c60-4c66-96a1-c11b651c1ad4
md"""
# Introduction

In this second part of the tutorial a machine learning algorithm will be applied to the 'glass' dataset, which we analyzed in part I.
"""

# ╔═╡ 48273931-e74b-4873-af65-42b526998c41
md"""
## MLJ - Machine Learning in Julia

For this purpose we use the [`MLJ`](https://alan-turing-institute.github.io/MLJ.jl/dev/) package. It is a so-called meta package which provides a common interface and common utility mechanisms for selecting, tuning, evaluating, composing and comparing of a vast number of ML models. At the moment of this writing, it offers 186 models.

MLJ is a [project](https://www.turing.ac.uk/research/research-projects/machine-learning-julia) at the *Turing Institute* in London in collaboration with the *University of Auckland* and other partners in science and industry. It started in 2019.
"""

# ╔═╡ 78f23ac8-7975-4aa5-8da0-fab4e0c165a1
md"""
## The next steps

Using a ML algorithm typically involves the following steps:

- Preparing the data, so that it gets the adequate structure for training and testing.
- Selecting an appropriate model which fits the data given and which produces the desired result (e.g. a classification or a clustering).
- Training the model choosen with a training dataset.
- Evaluating the trained model using a test dataset.

We will apply this ML workflow in the following section.
"""

# ╔═╡ aee28567-9a5d-471d-97f4-76639415219a
md"""
# The ML Workflow
"""

# ╔═╡ cf353269-6f66-49f9-8b84-5c81a7a981bd
md"""
## Preparing the data

Almost all ML models expect the training dataset to be in a tabular form. So the `DataFrame` data type which we used in part I is a good starting point.

In our case we have a so-called classification problem: We want the model to predict the correct glass type when given a set of values (namely our features RI and Na ... Ba; the independent attributes). The model will be trained using data for the independent attributes as well as the corresponding glass type which results in each case. This is called supervised learning.

For this sort of training we need the data of the independent attributes in tabular form and the resulting types in a separate list. The function `unpack` does this job. It splits the `glass` `DataFrame` in these two components (`XGlass` and `yGlass`). Apart from this it also shuffels the rows using a random number generator which gets initialized with the 'seed' `rng = 123`.
"""

# ╔═╡ b55d1e30-4408-4f52-923a-b44619037505
yGlass, XGlass = unpack(glass, ==(:Type), rng = 123)

# ╔═╡ a53006a4-3df1-4beb-b6b7-f9c2ab579e29
md"""
And indeed, we've got a vector and a table:
"""

# ╔═╡ e7a31074-60cf-4e9c-8097-25f398f23ff2
scitype(yGlass), scitype(XGlass)

# ╔═╡ af84d229-832a-48c8-ace2-defe1bf3b468
md"""
## Choosing a model
"""

# ╔═╡ 179712a4-d8c9-419a-a708-15918d1dfa6e
all_models = models()

# ╔═╡ 608af765-55b4-4412-b293-513a25e35cd8
md"""
Now we have to choose an appropriate model. The `models` function gives us a list of all models MLJ offers. There are currently $(length(all_models)) models in this list, which might be a bit overwhelming.
"""

# ╔═╡ 84fa53e7-f0f5-44c8-a976-b37f9356e9c0
compatible_models = models(matching(XGlass, yGlass))

# ╔═╡ b940caf9-c0fc-43ea-b97d-0df5463bc1a6
md"""
So we should narrow down the list a bit: First we want to see which models are basically 'compatible' with the data types of our dataset. All of our features have *continuous numerical* values and the resulting class (the glass type) is a *nominal* value.

Using `matching(XGlass, yGlass)` with `models` filters exactly those models which fulfill this requirement. This reduces the number considerably to $(length(compatible_models)):
"""

# ╔═╡ 7e41f4e3-a2ae-4b39-aeaf-06c900b83135
md"""
Well, apart from this rather technical 'compatibility' we have a few other expectations to our model: For this tutorial we want a model with high explainability, i.e. it should be easy to understand how it works when classifying some data. And ideally it should be implemented in pure Julia, as it is a tutorial about Julia and ML.

*Decision tree* classifiers have a high level of explainability. So let's have a look, if we can find some in this list which are implemented in Julia.

All models in MLJ are provided with comprehensive meta-information (as can be seen in the following example). E.g. the `docstring` contains a short textual description, `is_pure_julia` tells us, if it is implemented in pure Julia (or just some other programming language with a Julia wrapper around it) and finally each model has a `name`. We can use this meta-information to filter our list of compatible models:
"""

# ╔═╡ 41e0de70-b3a4-43fc-ae35-1bf10a71c371
filter(m -> m.is_pure_julia && contains(m.docstring * m.name, "DecisionTree"),
	compatible_models)

# ╔═╡ 40dc5eaf-fcc2-4803-a3d2-ce72126b77c9
md"""
So we could reduce our list to four candidates. The first and the 4th entry are advanced decision tree models. But we prefer a 'classic' one, which is easy to understand for the purpose of this tutorial. That's why we focus on candidates no. 2 & 3. The `docstring` of theses models contains an URL to their GitHub repository where we can also find the related documentation: No. 2 ist part of the [BetaML package](https://github.com/sylvaticus/BetaML.jl) and no. 3 can be found in the [DecisionTree package](https://github.com/bensadeghi/DecisionTree.jl).

Both implement the well known [CART algorithm](https://en.wikipedia.org/wiki/Decision_tree_learning) and are quite similar, but having a closer look at the documentation, the latter one seems to be more mature und offers a bit more functionality. So we choose this one.
"""

# ╔═╡ 724d8110-5a41-477b-9079-6dee4987ecd4
md"""
## Using a decision tree classifier
"""

# ╔═╡ f665c520-8e26-4762-bf79-f01dada35a47
md"""
A model can be loaded in MLJ using the `@load` macro. It returns a type definition (which we call here `MyDecisionTree`).
"""

# ╔═╡ deec9d4d-d701-46b2-b578-8a29623af096
MyDecisionTree = @load DecisionTreeClassifier pkg="DecisionTree"

# ╔═╡ 7ac6596a-d24b-4ae7-8339-7076cff64df0
md"""
In the next step we create an instance `dc` of this type. The instance contains all parameters with their default values. So we can see e.g. that the depth of the tree is not limited by default (`max_depth = -1`) and that it won't be pruned after creation (`post_prune = false`).
"""

# ╔═╡ a0f39794-7bfb-42bf-8525-6824d97c36b1
dc = MyDecisionTree()

# ╔═╡ 8a4a5e32-29c5-4d6e-88c9-82afa2ffd7b5
md"""
### Training the model
"""

# ╔═╡ fd954f51-857f-4a99-8029-0fe2b2723cda
md"""
We have now everything together to train our model. First we split our data into a *training data* set and a *test dataset* using a simple holdout strategy (in this case a 70:30 split). I.e. we lay 30% of our data aside for testing and use only 70% for training.

`partition` does this job and shuffels the rows using a random number generator (setting seed to `rng = 123`) before applying the split. As we already did a vertical split of the data into `XGlass` and `yGlass`, these components get splitted seperately into `Xtrain`/`Xtest` and `ytrain`/`ytest` respectively. We inform `partition` about this special situation by setting `multi = true`.
"""

# ╔═╡ 1b408007-5994-4f61-91ed-bb6f1a258114
(Xtrain, Xtest), (ytrain, ytest) =
	partition((XGlass, yGlass), 0.7, multi = true, rng = 123)

# ╔═╡ e2b31234-8ae3-492b-b61c-9d0bc38adec4
md"""
So `Xtrain, ytrain` contain now 70% of the 'glass' dataset and `Xtest, ytest` the remaining 30%.
"""

# ╔═╡ 9027b99d-2554-4341-b75a-8172932f239f
md"""
Next, we apply the following steps for training and testing:
- First the model and the training data get connected using a so-called *machine*.
- Then the training takes place by calling `fit!` on the machine.
- Afterwards the trained machine can be used to do predictions on new data (using `predict`).
"""

# ╔═╡ 81d68fa4-12dc-4183-8985-e7609febaf0c
md"""
For the sake of simplicity, we restrict the depth of the resulting decision tree to 3:
"""

# ╔═╡ 242c1317-df01-4859-bed5-dd2d3224cf50
dc.max_depth = 3

# ╔═╡ c67b3fb2-e9c7-49cc-a174-88c46c03030d
dc_mach = machine(dc, Xtrain, ytrain)

# ╔═╡ c465abe9-50dc-4080-88d2-fae9a6191eff
fit!(dc_mach)

# ╔═╡ 760d48c9-9d58-4b4c-9eee-d6df12aeec0f
md"""
### Predict and evaluate
"""

# ╔═╡ 78230dbe-87a9-453b-ae23-97ff4ab0d03d
md"""
So let's see how well that model predicts on the test dataset:
"""

# ╔═╡ e4f87f1a-44c1-4928-8d3c-af8f9a5fd2d5
yhat = predict(dc_mach, Xtest)

# ╔═╡ ad76082b-25bd-4a93-b590-2b5db3ea8245
md"""
The predictions in `yhat` don't look perhaps  as you would have expected: `yhat` isn't a list of predicted glass types, but a list of *distributions* stating the predicted probabilities for each possible glass type. This stems from the fact that the selected model (a `DecisionTreeClassifier`) is a *probabilistic model* (as can be seen from it's meta-information: `prediction_type = :probabilistic`). All probabilistic models in MLJ predict distributions (using `Distributions.jl`).
"""

# ╔═╡ 2c441c14-61da-43b5-ac1b-ee957769ce79
md"""
But it's easy to get a list of predicted classes (those with the highest probability within each distribution), just by calling `predict_mode` (instead of `predict`):
"""

# ╔═╡ 8341dc4e-8fd7-4bb5-b1bb-bc31e9a71e92
yhat_classes = predict_mode(dc_mach, Xtest)

# ╔═╡ db2384ea-918c-447a-8a32-28106f753755
md"""
On this basis we can check, how many classes have been predicted correctly by comparing `ytest` (the *correct* classes from the testset) to `yhat_classes` (the predicted classes from our classifier).

Note that we want to compare the *elements* of these two arrays with each other (not the arrays themselves). This can be achieved in Julia using the so-called *broadcast* mechanism which is denoted by using a dot in front of the function applied (in this case a comparison on equality using `==`). So `ytest .== yhat_classes` does an elementwise comparison resulting in a list of boolean values. Finally `count` counts the number of `true` values within this list.

"""

# ╔═╡ 5753000d-db74-4677-a074-ebd7c22692fd
correct_classes = count(ytest .== yhat_classes)

# ╔═╡ 8e0077ce-1905-4c8e-ba27-0f3a734ab394
accuracy = correct_classes / length(ytest)

# ╔═╡ cb0db38e-7124-4409-972a-a719012a09ad
md"""
Using the result of the count we can see, that $(correct_classes) of the $(length(ytest)) instances in the testset have been predicted correctly. This is a rate of $(accuracy * 100)%.
"""

# ╔═╡ e199f8cc-b2de-4bd5-a077-41d683923cf6
md"""
### Print the decision tree
"""

# ╔═╡ 3d934675-e00c-4cc9-9073-f11baf52109d
md"""
Now it would be nice, if we could see how the decision tree, that has been constructed by our `DecisionTreeClassifier`, looks like. Then we could e.g. check which attributes have been used for branching and at which thresholds that took place.
"""

# ╔═╡ e574f5db-b7cf-4d4f-9803-026a43337f6d
md"""
Fortunately there is a built-in function for this purpose. `report` is a generic function that can be called on a trained machine and delivers (depending on the model used) all relevant information about the trained machine. In case of a decision tree, this 'relevant information' is (among other things) a `TreePrinter` object, which can be called using `print_tree()`.
"""

# ╔═╡ c3a70f7e-bf69-41f8-aecc-9c429d5de1c1
report(dc_mach).print_tree()

# ╔═╡ 9afbdf3c-9bdc-47aa-863a-8cee05013dba
names(glass)

# ╔═╡ 4624e243-952c-4549-bfc4-def2da8dfe4a
describe(glass)

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
ARFFFiles = "da404889-ca92-49ff-9e8b-0aa6b4d38dc8"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
Downloads = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
MLJ = "add582a8-e3ab-11e8-2d5e-e98b27df1bc7"
MLJDecisionTreeInterface = "c6f25543-311c-4c74-83dc-3ea6d1015661"
PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
ScientificTypes = "321657f4-b219-11e9-178b-2701a2544e81"
StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
StatsPlots = "f3b207a7-027a-5e70-b257-86293d7955fd"

[compat]
ARFFFiles = "~1.4.1"
DataFrames = "~1.3.2"
MLJ = "~0.17.3"
MLJDecisionTreeInterface = "~0.2.1"
PlutoUI = "~0.7.37"
ScientificTypes = "~3.0.0"
StatsBase = "~0.33.16"
StatsPlots = "~0.14.33"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

julia_version = "1.8.0-DEV.1390"
manifest_format = "2.0"
project_hash = "a72ce86d6cef0527ebe0a1cb99c8cf512f0741ce"

[[deps.ARFFFiles]]
deps = ["CategoricalArrays", "Dates", "Parsers", "Tables"]
git-tree-sha1 = "e8c8e0a2be6eb4f56b1672e46004463033daa409"
uuid = "da404889-ca92-49ff-9e8b-0aa6b4d38dc8"
version = "1.4.1"

[[deps.AbstractFFTs]]
deps = ["ChainRulesCore", "LinearAlgebra"]
git-tree-sha1 = "6f1d9bc1c08f9f4a8fa92e3ea3cb50153a1b40d4"
uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c"
version = "1.1.0"

[[deps.AbstractPlutoDingetjes]]
deps = ["Pkg"]
git-tree-sha1 = "8eaf9f1b4921132a4cff3f36a1d9ba923b14a481"
uuid = "6e696c72-6542-2067-7265-42206c756150"
version = "1.1.4"

[[deps.Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.3"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
version = "1.1.1"

[[deps.Arpack]]
deps = ["Arpack_jll", "Libdl", "LinearAlgebra", "Logging"]
git-tree-sha1 = "91ca22c4b8437da89b030f08d71db55a379ce958"
uuid = "7d9fca2a-8960-54d3-9f78-7d1dccf2cb97"
version = "0.5.3"

[[deps.Arpack_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "OpenBLAS_jll", "Pkg"]
git-tree-sha1 = "5ba6c757e8feccf03a1554dfaf3e26b3cfc7fd5e"
uuid = "68821587-b530-5797-8361-c406ea357684"
version = "3.5.1+1"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[deps.AxisAlgorithms]]
deps = ["LinearAlgebra", "Random", "SparseArrays", "WoodburyMatrices"]
git-tree-sha1 = "66771c8d21c8ff5e3a93379480a2307ac36863f7"
uuid = "13072b0f-2c55-5437-9ae7-d433b7a33950"
version = "1.0.1"

[[deps.BSON]]
git-tree-sha1 = "306bb5574b0c1c56d7e1207581516c557d105cad"
uuid = "fbb218c0-5317-5bc6-957e-2ee96dd4b1f0"
version = "0.3.5"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[deps.Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "4b859a208b2397a7a623a03449e4636bdb17bcf2"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+1"

[[deps.Calculus]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad"
uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9"
version = "0.5.1"

[[deps.CategoricalArrays]]
deps = ["DataAPI", "Future", "Missings", "Printf", "Requires", "Statistics", "Unicode"]
git-tree-sha1 = "5196120341b6dfe3ee5f33cf97392a05d6fe80d0"
uuid = "324d7699-5711-5eae-9e2f-1d82baa6b597"
version = "0.10.4"

[[deps.CategoricalDistributions]]
deps = ["CategoricalArrays", "Distributions", "Missings", "OrderedCollections", "Random", "ScientificTypesBase", "UnicodePlots"]
git-tree-sha1 = "8c340dc71d2dc9177b1f701726d08d2255d2d811"
uuid = "af321ab8-2d2e-40a6-b165-3d674595d28e"
version = "0.1.5"

[[deps.ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "c9a6160317d1abe9c44b3beb367fd448117679ca"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.13.0"

[[deps.ChangesOfVariables]]
deps = ["ChainRulesCore", "LinearAlgebra", "Test"]
git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1"
uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
version = "0.1.2"

[[deps.Clustering]]
deps = ["Distances", "LinearAlgebra", "NearestNeighbors", "Printf", "SparseArrays", "Statistics", "StatsBase"]
git-tree-sha1 = "75479b7df4167267d75294d14b58244695beb2ac"
uuid = "aaaa29a8-35af-508c-8bc3-b662a17a0fe5"
version = "0.14.2"

[[deps.CodecZlib]]
deps = ["TranscodingStreams", "Zlib_jll"]
git-tree-sha1 = "ded953804d019afa9a3f98981d99b33e3db7b6da"
uuid = "944b1d66-785c-5afd-91f1-9de20f533193"
version = "0.7.0"

[[deps.ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "12fc73e5e0af68ad3137b886e3f7c1eacfca2640"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.17.1"

[[deps.ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[deps.Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[deps.Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "96b0bc6c52df76506efc8a441c6cf1adcb1babc4"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.42.0"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
version = "0.5.0+0"

[[deps.ComputationalResources]]
git-tree-sha1 = "52cb3ec90e8a8bea0e62e275ba577ad0f74821f7"
uuid = "ed09eef8-17a6-5b46-8889-db040fac31e3"
version = "0.3.2"

[[deps.ConstructionBase]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "f74e9d5388b8620b4cee35d4c5a618dd4dc547f4"
uuid = "187b0558-2788-49d3-abe0-74a17ed4e7c9"
version = "1.3.0"

[[deps.Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[deps.Crayons]]
git-tree-sha1 = "249fe38abf76d48563e2f4556bebd215aa317e15"
uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f"
version = "4.1.1"

[[deps.DataAPI]]
git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.9.0"

[[deps.DataFrames]]
deps = ["Compat", "DataAPI", "Future", "InvertedIndices", "IteratorInterfaceExtensions", "LinearAlgebra", "Markdown", "Missings", "PooledArrays", "PrettyTables", "Printf", "REPL", "Reexport", "SortingAlgorithms", "Statistics", "TableTraits", "Tables", "Unicode"]
git-tree-sha1 = "ae02104e835f219b8930c7664b8012c93475c340"
uuid = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
version = "1.3.2"

[[deps.DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.11"

[[deps.DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[deps.DataValues]]
deps = ["DataValueInterfaces", "Dates"]
git-tree-sha1 = "d88a19299eba280a6d062e135a43f00323ae70bf"
uuid = "e7dc6d0d-1eca-5fa6-8ad6-5aecde8b7ea5"
version = "0.4.13"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[deps.DecisionTree]]
deps = ["DelimitedFiles", "Distributed", "LinearAlgebra", "Random", "ScikitLearnBase", "Statistics", "Test"]
git-tree-sha1 = "123adca1e427dc8abc5eec5040644e7842d53c92"
uuid = "7806a523-6efd-50cb-b5f6-3fa6f1930dbb"
version = "0.10.11"

[[deps.DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[deps.DensityInterface]]
deps = ["InverseFunctions", "Test"]
git-tree-sha1 = "80c3e8639e3353e5d2912fb3a1916b8455e2494b"
uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
version = "0.4.0"

[[deps.Distances]]
deps = ["LinearAlgebra", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "3258d0659f812acde79e8a74b11f17ac06d0ca04"
uuid = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
version = "0.10.7"

[[deps.Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[deps.Distributions]]
deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Test"]
git-tree-sha1 = "43ea1b7ab7936920a1170d6a35f05a1f9f495216"
uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
version = "0.25.51"

[[deps.DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.6"

[[deps.Downloads]]
deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
version = "1.6.0"

[[deps.DualNumbers]]
deps = ["Calculus", "NaNMath", "SpecialFunctions"]
git-tree-sha1 = "90b158083179a6ccbce2c7eb1446d5bf9d7ae571"
uuid = "fa6b7ba4-c1ee-5f82-b5fc-ecf0adba8f74"
version = "0.6.7"

[[deps.EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.2.3+0"

[[deps.EarlyStopping]]
deps = ["Dates", "Statistics"]
git-tree-sha1 = "98fdf08b707aaf69f524a6cd0a67858cefe0cfb6"
uuid = "792122b4-ca99-40de-a6bc-6742525f08b6"
version = "0.3.0"

[[deps.Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ae13fcbc7ab8f16b0856729b050ef0c446aa3492"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.4.4+0"

[[deps.FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[deps.FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.4.0+0"

[[deps.FFTW]]
deps = ["AbstractFFTs", "FFTW_jll", "LinearAlgebra", "MKL_jll", "Preferences", "Reexport"]
git-tree-sha1 = "505876577b5481e50d089c1c68899dfb6faebc62"
uuid = "7a1cc6ca-52ef-59f5-83cd-3a7055c09341"
version = "1.4.6"

[[deps.FFTW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c6033cc3892d0ef5bb9cd29b7f2f0331ea5184ea"
uuid = "f5851436-0d7a-5f13-b9de-f02708fd171a"
version = "3.3.10+0"

[[deps.FilePathsBase]]
deps = ["Compat", "Dates", "Mmap", "Printf", "Test", "UUIDs"]
git-tree-sha1 = "04d13bfa8ef11720c24e4d840c0033d145537df7"
uuid = "48062228-2e41-5def-b9a4-89aafe57970f"
version = "0.9.17"

[[deps.FileWatching]]
uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee"

[[deps.FillArrays]]
deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"]
git-tree-sha1 = "0dbc5b9683245f905993b51d2814202d75b34f1a"
uuid = "1a297f60-69ca-5386-bcde-b61e274b549b"
version = "0.13.1"

[[deps.FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[deps.Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[deps.Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[deps.FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[deps.FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[deps.Future]]
deps = ["Random"]
uuid = "9fa8497b-333b-5362-9e8d-4d0656e87820"

[[deps.GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "51d2dfe8e590fbd74e7a842cf6d13d8a2f45dc01"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.6+0"

[[deps.GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "RelocatableFolders", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "9f836fb62492f4b0f0d3b06f55983f2704ed0883"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.64.0"

[[deps.GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "a6c850d77ad5118ad3be4bd188919ce97fffac47"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.64.0+0"

[[deps.GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "83ea630384a13fc4f002b77690bc0afeb4255ac9"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.2"

[[deps.Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[deps.Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "a32d672ac2c967f3deb8a81d828afc739c838a06"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+2"

[[deps.Graphite2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011"
uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472"
version = "1.3.14+0"

[[deps.Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[deps.HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "0fa77022fe4b511826b39c894c90daf5fce3334a"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.17"

[[deps.HarfBuzz_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"]
git-tree-sha1 = "129acf094d168394e80ee1dc4bc06ec835e510a3"
uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566"
version = "2.8.1+1"

[[deps.HypergeometricFunctions]]
deps = ["DualNumbers", "LinearAlgebra", "SpecialFunctions", "Test"]
git-tree-sha1 = "65e4589030ef3c44d3b90bdc5aac462b4bb05567"
uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a"
version = "0.3.8"

[[deps.Hyperscript]]
deps = ["Test"]
git-tree-sha1 = "8d511d5b81240fc8e6802386302675bdf47737b9"
uuid = "47d2ed2b-36de-50cf-bf87-49c2cf4b8b91"
version = "0.0.4"

[[deps.HypertextLiteral]]
git-tree-sha1 = "2b078b5a615c6c0396c77810d92ee8c6f470d238"
uuid = "ac1192a8-f4b3-4bfe-ba22-af5b92cd3ab2"
version = "0.9.3"

[[deps.IOCapture]]
deps = ["Logging", "Random"]
git-tree-sha1 = "f7be53659ab06ddc986428d3a9dcc95f6fa6705a"
uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
version = "0.2.2"

[[deps.IniFile]]
git-tree-sha1 = "f550e6e32074c939295eb5ea6de31849ac2c9625"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.1"

[[deps.IntelOpenMP_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d979e54b71da82f3a65b62553da4fc3d18c9004c"
uuid = "1d5cc7b8-4909-519e-a0f8-d0f5ad9712d0"
version = "2018.0.3+2"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[deps.Interpolations]]
deps = ["AxisAlgorithms", "ChainRulesCore", "LinearAlgebra", "OffsetArrays", "Random", "Ratios", "Requires", "SharedArrays", "SparseArrays", "StaticArrays", "WoodburyMatrices"]
git-tree-sha1 = "b15fc0a95c564ca2e0a7ae12c1f095ca848ceb31"
uuid = "a98d9a8b-a2ab-59e6-89dd-64a1c18fca59"
version = "0.13.5"

[[deps.InverseFunctions]]
deps = ["Test"]
git-tree-sha1 = "91b5dcf362c5add98049e6c29ee756910b03051d"
uuid = "3587e190-3f89-42d0-90ee-14403ec27112"
version = "0.1.3"

[[deps.InvertedIndices]]
git-tree-sha1 = "bee5f1ef5bf65df56bdd2e40447590b272a5471f"
uuid = "41ab1584-1d38-5bbf-9106-f11c6c58b48f"
version = "1.1.0"

[[deps.IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[deps.IterTools]]
git-tree-sha1 = "fa6287a4469f5e048d763df38279ee729fbd44e5"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.4.0"

[[deps.IterationControl]]
deps = ["EarlyStopping", "InteractiveUtils"]
git-tree-sha1 = "83c84b7b87d3063e48a909a86c3c5bf4c3521962"
uuid = "b3c1a2ee-3fec-4384-bf48-272ea71de57c"
version = "0.5.2"

[[deps.IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[deps.JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.4.1"

[[deps.JLSO]]
deps = ["BSON", "CodecZlib", "FilePathsBase", "Memento", "Pkg", "Serialization"]
git-tree-sha1 = "e00feb9d56e9e8518e0d60eef4d1040b282771e2"
uuid = "9da8a3cd-07a3-59c0-a743-3fdc52c30d11"
version = "2.6.0"

[[deps.JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "3c837543ddb02250ef42f4738347454f95079d4e"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.3"

[[deps.JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b53380851c6e6664204efb2e62cd24fa5c47e4ba"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.2+0"

[[deps.KernelDensity]]
deps = ["Distributions", "DocStringExtensions", "FFTW", "Interpolations", "StatsBase"]
git-tree-sha1 = "591e8dc09ad18386189610acafb970032c519707"
uuid = "5ab0869b-81aa-558d-bb23-cbf5423bbe9b"
version = "0.6.3"

[[deps.LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[deps.LERC_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "bf36f528eec6634efc60d7ec062008f171071434"
uuid = "88015f11-f218-50d7-93a8-a6af411a945d"
version = "3.0.0+1"

[[deps.LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[deps.LaTeXStrings]]
git-tree-sha1 = "f2355693d6778a178ade15952b7ac47a4ff97996"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.3.0"

[[deps.Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "4f00cc36fede3c04b8acf9b2e2763decfdcecfa6"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.13"

[[deps.LatinHypercubeSampling]]
deps = ["Random", "StableRNGs", "StatsBase", "Test"]
git-tree-sha1 = "42938ab65e9ed3c3029a8d2c58382ca75bdab243"
uuid = "a5e1c1ea-c99a-51d3-a14d-a9a37257b02d"
version = "1.8.0"

[[deps.LazyArtifacts]]
deps = ["Artifacts", "Pkg"]
uuid = "4af54fe1-eca0-43a8-85a7-787d91b784e3"

[[deps.LearnBase]]
git-tree-sha1 = "a0d90569edd490b82fdc4dc078ea54a5a800d30a"
uuid = "7f8f8fb0-2700-5f03-b4bd-41f8cfc144b6"
version = "0.4.1"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
version = "0.6.3"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
version = "7.73.0+4"

[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
version = "1.9.1+2"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[deps.Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "0b4a5d71f3e5200a7dff793393e09dfc2d874290"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+1"

[[deps.Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[deps.Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[deps.Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[deps.Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[deps.Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[deps.Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "LERC_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "c9551dd26e31ab17b86cbd00c2ede019c08758eb"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+1"

[[deps.Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[deps.LinearAlgebra]]
deps = ["Libdl", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[deps.LogExpFunctions]]
deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "58f25e56b706f95125dcb796f39e1fb01d913a71"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.10"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[deps.LossFunctions]]
deps = ["InteractiveUtils", "LearnBase", "Markdown", "RecipesBase", "StatsBase"]
git-tree-sha1 = "0f057f6ea90a84e73a8ef6eebb4dc7b5c330020f"
uuid = "30fc2ffe-d236-52d8-8643-a9d8f7c094a7"
version = "0.7.2"

[[deps.MKL_jll]]
deps = ["Artifacts", "IntelOpenMP_jll", "JLLWrappers", "LazyArtifacts", "Libdl", "Pkg"]
git-tree-sha1 = "e595b205efd49508358f7dc670a940c790204629"
uuid = "856f044c-d86e-5d09-b602-aeab76dc8ba7"
version = "2022.0.0+0"

[[deps.MLJ]]
deps = ["CategoricalArrays", "ComputationalResources", "Distributed", "Distributions", "LinearAlgebra", "MLJBase", "MLJEnsembles", "MLJIteration", "MLJModels", "MLJSerialization", "MLJTuning", "OpenML", "Pkg", "ProgressMeter", "Random", "ScientificTypes", "Statistics", "StatsBase", "Tables"]
git-tree-sha1 = "ecd156a5494894ea125548ee58226541ee368329"
uuid = "add582a8-e3ab-11e8-2d5e-e98b27df1bc7"
version = "0.17.3"

[[deps.MLJBase]]
deps = ["CategoricalArrays", "CategoricalDistributions", "ComputationalResources", "Dates", "DelimitedFiles", "Distributed", "Distributions", "InteractiveUtils", "InvertedIndices", "LinearAlgebra", "LossFunctions", "MLJModelInterface", "Missings", "OrderedCollections", "Parameters", "PrettyTables", "ProgressMeter", "Random", "ScientificTypes", "StatisticalTraits", "Statistics", "StatsBase", "Tables"]
git-tree-sha1 = "2e41aab645157a8d9b53c478672459317d0a3ad9"
uuid = "a7f614a8-145f-11e9-1d2a-a57a1082229d"
version = "0.19.8"

[[deps.MLJDecisionTreeInterface]]
deps = ["DecisionTree", "MLJModelInterface", "Random", "Tables"]
git-tree-sha1 = "8cf2bb326b2d7d8a81c2b14e709dc2aeeb25bbc6"
uuid = "c6f25543-311c-4c74-83dc-3ea6d1015661"
version = "0.2.1"

[[deps.MLJEnsembles]]
deps = ["CategoricalArrays", "CategoricalDistributions", "ComputationalResources", "Distributed", "Distributions", "MLJBase", "MLJModelInterface", "ProgressMeter", "Random", "ScientificTypesBase", "StatsBase"]
git-tree-sha1 = "4279437ccc8ece8f478ded5139334b888dcce631"
uuid = "50ed68f4-41fd-4504-931a-ed422449fee0"
version = "0.2.0"

[[deps.MLJIteration]]
deps = ["IterationControl", "MLJBase", "Random"]
git-tree-sha1 = "9ea78184700a54ce45abea4c99478aa5261ed74f"
uuid = "614be32b-d00c-4edb-bd02-1eb411ab5e55"
version = "0.4.5"

[[deps.MLJModelInterface]]
deps = ["Random", "ScientificTypesBase", "StatisticalTraits"]
git-tree-sha1 = "74d7fb54c306af241c5f9d4816b735cb4051e125"
uuid = "e80e1ace-859a-464e-9ed9-23947d8ae3ea"
version = "1.4.2"

[[deps.MLJModels]]
deps = ["CategoricalArrays", "CategoricalDistributions", "Dates", "Distances", "Distributions", "InteractiveUtils", "LinearAlgebra", "MLJModelInterface", "Markdown", "OrderedCollections", "Parameters", "Pkg", "PrettyPrinting", "REPL", "Random", "ScientificTypes", "StatisticalTraits", "Statistics", "StatsBase", "Tables"]
git-tree-sha1 = "f6f971456d38f020ce476283e1c27f2481bf99ac"
uuid = "d491faf4-2d78-11e9-2867-c94bc002c0b7"
version = "0.15.6"

[[deps.MLJSerialization]]
deps = ["IterationControl", "JLSO", "MLJBase", "MLJModelInterface"]
git-tree-sha1 = "cc5877ad02ef02e273d2622f0d259d628fa61cd0"
uuid = "17bed46d-0ab5-4cd4-b792-a5c4b8547c6d"
version = "1.1.3"

[[deps.MLJTuning]]
deps = ["ComputationalResources", "Distributed", "Distributions", "LatinHypercubeSampling", "MLJBase", "ProgressMeter", "Random", "RecipesBase"]
git-tree-sha1 = "a443cc088158b949876d7038a1aa37cfc8c5509b"
uuid = "03970b2e-30c4-11ea-3135-d1576263f10f"
version = "0.6.16"

[[deps.MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.9"

[[deps.MarchingCubes]]
deps = ["StaticArrays"]
git-tree-sha1 = "5f768e0a0c3875df386be4c036f78c8bd4b1a9b6"
uuid = "299715c1-40a9-479a-aaf9-4a633d36f717"
version = "0.1.2"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[deps.MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
version = "2.24.0+2"

[[deps.Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[deps.Memento]]
deps = ["Dates", "Distributed", "Requires", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "9b0b0dbf419fbda7b383dc12d108621d26eeb89f"
uuid = "f28f55f0-a522-5efc-85c2-fe41dfb9b2d9"
version = "1.3.0"

[[deps.Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[deps.Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
version = "2020.7.22"

[[deps.MultivariateStats]]
deps = ["Arpack", "LinearAlgebra", "SparseArrays", "Statistics", "StatsAPI", "StatsBase"]
git-tree-sha1 = "7008a3412d823e29d370ddc77411d593bd8a3d03"
uuid = "6f286f6a-111f-5878-ab1e-185364afe411"
version = "0.9.1"

[[deps.NaNMath]]
git-tree-sha1 = "737a5957f387b17e74d4ad2f440eb330b39a62c5"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "1.0.0"

[[deps.NearestNeighbors]]
deps = ["Distances", "StaticArrays"]
git-tree-sha1 = "16baacfdc8758bc374882566c9187e785e85c2f0"
uuid = "b8a86587-4115-5ab1-83bc-aa920d37bbce"
version = "0.4.9"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
version = "1.2.0"

[[deps.Observables]]
git-tree-sha1 = "fe29afdef3d0c4a8286128d4e45cc50621b1e43d"
uuid = "510215fc-4207-5dde-b226-833fc4488ee2"
version = "0.4.0"

[[deps.OffsetArrays]]
deps = ["Adapt"]
git-tree-sha1 = "043017e0bdeff61cfbb7afeb558ab29536bbb5ed"
uuid = "6fe1bfb0-de20-5000-8ca7-80f57d26f881"
version = "1.10.8"

[[deps.Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "887579a3eb005446d514ab7aeac5d1d027658b8f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+1"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
version = "0.3.17+2"

[[deps.OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"
version = "0.8.1+0"

[[deps.OpenML]]
deps = ["ARFFFiles", "HTTP", "JSON", "Markdown", "Pkg"]
git-tree-sha1 = "06080992e86a93957bfe2e12d3181443cedf2400"
uuid = "8b6db2d4-7670-4922-a472-f9537c81ab66"
version = "0.2.0"

[[deps.OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ab05aa4cc89736e95915b01e7279e61b1bfe33b8"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.14+0"

[[deps.OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[deps.Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[deps.OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[deps.PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[deps.PDMats]]
deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"]
git-tree-sha1 = "e8185b83b9fc56eb6456200e873ce598ebc7f262"
uuid = "90014a1f-27ba-587c-ab20-58faa44d9150"
version = "0.11.7"

[[deps.Parameters]]
deps = ["OrderedCollections", "UnPack"]
git-tree-sha1 = "34c0e9ad262e5f7fc75b10a9952ca7692cfc5fbe"
uuid = "d96e819e-fc66-5662-9728-84c9c7592b0a"
version = "0.12.3"

[[deps.Parsers]]
deps = ["Dates"]
git-tree-sha1 = "85b5da0fa43588c75bb1ff986493443f821c70b7"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.2.3"

[[deps.Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
version = "1.8.0"

[[deps.PlotThemes]]
deps = ["PlotUtils", "Requires", "Statistics"]
git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "2.0.1"

[[deps.PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "bb16469fd5224100e422f0b027d26c5a25de1200"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.2.0"

[[deps.Plots]]
deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "Pkg", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs", "UnicodeFun", "Unzip"]
git-tree-sha1 = "9213b4c18b57b7020ee20f33a4ba49eb7bef85e0"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.27.0"

[[deps.PlutoUI]]
deps = ["AbstractPlutoDingetjes", "Base64", "ColorTypes", "Dates", "Hyperscript", "HypertextLiteral", "IOCapture", "InteractiveUtils", "JSON", "Logging", "Markdown", "Random", "Reexport", "UUIDs"]
git-tree-sha1 = "bf0a1121af131d9974241ba53f601211e9303a9e"
uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
version = "0.7.37"

[[deps.PooledArrays]]
deps = ["DataAPI", "Future"]
git-tree-sha1 = "db3a23166af8aebf4db5ef87ac5b00d36eb771e2"
uuid = "2dfb63ee-cc39-5dd5-95bd-886bf059d720"
version = "1.4.0"

[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "d3538e7f8a790dc8903519090857ef8e1283eecd"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.5"

[[deps.PrettyPrinting]]
git-tree-sha1 = "4be53d093e9e37772cc89e1009e8f6ad10c4681b"
uuid = "54e16d92-306c-5ea0-a30b-337be88ac337"
version = "0.4.0"

[[deps.PrettyTables]]
deps = ["Crayons", "Formatting", "Markdown", "Reexport", "Tables"]
git-tree-sha1 = "dfb54c4e414caa595a1f2ed759b160f5a3ddcba5"
uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
version = "1.3.1"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[deps.ProgressMeter]]
deps = ["Distributed", "Printf"]
git-tree-sha1 = "afadeba63d90ff223a6a48d2009434ecee2ec9e8"
uuid = "92933f4c-e287-5a05-a399-4b506db050ca"
version = "1.7.1"

[[deps.Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+0"

[[deps.QuadGK]]
deps = ["DataStructures", "LinearAlgebra"]
git-tree-sha1 = "78aadffb3efd2155af139781b8a8df1ef279ea39"
uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
version = "2.4.2"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[deps.Random]]
deps = ["SHA", "Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[deps.Ratios]]
deps = ["Requires"]
git-tree-sha1 = "dc84268fe0e3335a62e315a3a7cf2afa7178a734"
uuid = "c84ed2f1-dad5-54f0-aa8e-dbefe2724439"
version = "0.4.3"

[[deps.RecipesBase]]
git-tree-sha1 = "6bf3f380ff52ce0832ddd3a2a7b9538ed1bcca7d"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.2.1"

[[deps.RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "995a812c6f7edea7527bb570f0ac39d0fb15663c"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.5.1"

[[deps.Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[deps.RelocatableFolders]]
deps = ["SHA", "Scratch"]
git-tree-sha1 = "cdbd3b1338c72ce29d9584fdbe9e9b70eeb5adca"
uuid = "05181044-ff0b-4ac5-8273-598c1e38db00"
version = "0.1.3"

[[deps.Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[deps.Rmath]]
deps = ["Random", "Rmath_jll"]
git-tree-sha1 = "bf3188feca147ce108c76ad82c2792c57abe7b1f"
uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa"
version = "0.7.0"

[[deps.Rmath_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "68db32dff12bb6127bac73c209881191bf0efbb7"
uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f"
version = "0.3.0+0"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
version = "0.7.0"

[[deps.ScientificTypes]]
deps = ["CategoricalArrays", "ColorTypes", "Dates", "Distributions", "PrettyTables", "Reexport", "ScientificTypesBase", "StatisticalTraits", "Tables"]
git-tree-sha1 = "ba70c9a6e4c81cc3634e3e80bb8163ab5ef57eb8"
uuid = "321657f4-b219-11e9-178b-2701a2544e81"
version = "3.0.0"

[[deps.ScientificTypesBase]]
git-tree-sha1 = "a8e18eb383b5ecf1b5e6fc237eb39255044fd92b"
uuid = "30f210dd-8aff-4c5f-94ba-8e64358c1161"
version = "3.0.0"

[[deps.ScikitLearnBase]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "7877e55c1523a4b336b433da39c8e8c08d2f221f"
uuid = "6e75b9c4-186b-50bd-896f-2d2496a4843e"
version = "0.5.0"

[[deps.Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[deps.SentinelArrays]]
deps = ["Dates", "Random"]
git-tree-sha1 = "6a2f7d70512d205ca8c7ee31bfa9f142fe74310c"
uuid = "91c51154-3ec4-41a3-a24f-3f23e20d615c"
version = "1.3.12"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[deps.SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[deps.Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[deps.SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[deps.SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[deps.SpecialFunctions]]
deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "5ba658aeecaaf96923dce0da9e703bd1fe7666f9"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.1.4"

[[deps.StableRNGs]]
deps = ["Random", "Test"]
git-tree-sha1 = "3be7d49667040add7ee151fefaf1f8c04c8c8276"
uuid = "860ef19b-820b-49d6-a774-d7a799459cd3"
version = "1.0.0"

[[deps.StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "6976fab022fea2ffea3d945159317556e5dad87c"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.4.2"

[[deps.StatisticalTraits]]
deps = ["ScientificTypesBase"]
git-tree-sha1 = "271a7fea12d319f23d55b785c51f6876aadb9ac0"
uuid = "64bff920-2084-43da-a3e6-9bb72801c0c9"
version = "3.0.0"

[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[deps.StatsAPI]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "c3d8ba7f3fa0625b062b82853a7d5229cb728b6b"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.2.1"

[[deps.StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "8977b17906b0a1cc74ab2e3a05faa16cf08a8291"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.16"

[[deps.StatsFuns]]
deps = ["ChainRulesCore", "HypergeometricFunctions", "InverseFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"]
git-tree-sha1 = "25405d7016a47cf2bd6cd91e66f4de437fd54a07"
uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c"
version = "0.9.16"

[[deps.StatsPlots]]
deps = ["AbstractFFTs", "Clustering", "DataStructures", "DataValues", "Distributions", "Interpolations", "KernelDensity", "LinearAlgebra", "MultivariateStats", "Observables", "Plots", "RecipesBase", "RecipesPipeline", "Reexport", "StatsBase", "TableOperations", "Tables", "Widgets"]
git-tree-sha1 = "4d9c69d65f1b270ad092de0abe13e859b8c55cad"
uuid = "f3b207a7-027a-5e70-b257-86293d7955fd"
version = "0.14.33"

[[deps.StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "57617b34fa34f91d536eb265df67c2d4519b8b98"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.5"

[[deps.SuiteSparse]]
deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"]
uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
version = "1.0.0"

[[deps.TableOperations]]
deps = ["SentinelArrays", "Tables", "Test"]
git-tree-sha1 = "e383c87cf2a1dc41fa30c093b2a19877c83e1bc1"
uuid = "ab02a1b2-a7df-11e8-156e-fb1833f50b87"
version = "1.2.0"

[[deps.TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[deps.Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "OrderedCollections", "TableTraits", "Test"]
git-tree-sha1 = "5ce79ce186cc678bbb5c5681ca3379d1ddae11a1"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.7.0"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
version = "1.10.0"

[[deps.Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[deps.TranscodingStreams]]
deps = ["Random", "Test"]
git-tree-sha1 = "216b95ea110b5972db65aa90f88d8d89dcb8851c"
uuid = "3bb67fe8-82b1-5028-8e26-92a6c54297fa"
version = "0.9.6"

[[deps.URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[deps.UnPack]]
git-tree-sha1 = "387c1f73762231e86e0c9c5443ce3b4a0a9a0c2b"
uuid = "3a884ed6-31ef-47d7-9d2a-63182c4928ed"
version = "1.0.2"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[deps.UnicodeFun]]
deps = ["REPL"]
git-tree-sha1 = "53915e50200959667e78a92a418594b428dffddf"
uuid = "1cfade01-22cf-5700-b092-accc4b62d6e1"
version = "0.4.1"

[[deps.UnicodePlots]]
deps = ["Contour", "Crayons", "Dates", "LinearAlgebra", "MarchingCubes", "NaNMath", "SparseArrays", "StaticArrays", "StatsBase", "Unitful"]
git-tree-sha1 = "1785494cb9484f9ab05bbc9d81a2d4de4341eb39"
uuid = "b8865327-cd53-5732-bb35-84acbb429228"
version = "2.9.0"

[[deps.Unitful]]
deps = ["ConstructionBase", "Dates", "LinearAlgebra", "Random"]
git-tree-sha1 = "b649200e887a487468b71821e2644382699f1b0f"
uuid = "1986cc42-f94f-5a68-af5c-568840ba703d"
version = "1.11.0"

[[deps.Unzip]]
git-tree-sha1 = "34db80951901073501137bdbc3d5a8e7bbd06670"
uuid = "41fe7b60-77ed-43a1-b4f0-825fd5a5650d"
version = "0.1.2"

[[deps.Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[deps.Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4528479aa01ee1b3b4cd0e6faef0e04cf16466da"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.25.0+0"

[[deps.Widgets]]
deps = ["Colors", "Dates", "Observables", "OrderedCollections"]
git-tree-sha1 = "505c31f585405fc375d99d02588f6ceaba791241"
uuid = "cc8bc4a8-27d6-5769-a93b-9d913e69aa62"
version = "0.6.5"

[[deps.WoodburyMatrices]]
deps = ["LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "de67fa59e33ad156a590055375a30b23c40299d3"
uuid = "efce3f68-66dc-5838-9240-27a6d6f5f9b6"
version = "0.5.5"

[[deps.XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[deps.XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[deps.Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[deps.Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[deps.Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[deps.Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[deps.Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[deps.Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[deps.Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[deps.Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[deps.Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[deps.Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[deps.Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[deps.Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[deps.Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[deps.Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[deps.Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[deps.Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[deps.Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[deps.Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[deps.Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
version = "1.2.12+1"

[[deps.Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e45044cd873ded54b6a5bac0eb5c971392cf1927"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.2+0"

[[deps.libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.15.1+0"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
version = "4.0.0+0"

[[deps.libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "2.0.2+0"

[[deps.libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[deps.libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "b910cb81ef3fe6e78bf6acee440bda86fd6ae00c"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+1"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
version = "1.41.0+1"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
version = "16.2.1+1"

[[deps.x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2021.5.5+0"

[[deps.x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.5.0+0"

[[deps.xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╟─42a4863a-7955-11ec-117a-dbb08de9f921
# ╟─2129f49d-9e19-4fa1-b878-22b3c8b66a90
# ╟─4ed56657-4dab-4fe6-881f-12d1ae32a83a
# ╟─1a86389d-cae3-4848-bef1-b0ef1fe1ba7a
# ╠═280b6629-e391-459d-99fb-26a1b483073b
# ╟─377ae933-4d00-40cf-9355-332065094f60
# ╠═031cf26d-4fde-4a8a-a7d2-f49200c2fab7
# ╠═c5e832ea-4a18-430e-8d0f-305da07e3711
# ╟─116d483d-6c60-4c66-96a1-c11b651c1ad4
# ╟─48273931-e74b-4873-af65-42b526998c41
# ╟─78f23ac8-7975-4aa5-8da0-fab4e0c165a1
# ╟─aee28567-9a5d-471d-97f4-76639415219a
# ╟─cf353269-6f66-49f9-8b84-5c81a7a981bd
# ╠═b55d1e30-4408-4f52-923a-b44619037505
# ╟─a53006a4-3df1-4beb-b6b7-f9c2ab579e29
# ╠═e7a31074-60cf-4e9c-8097-25f398f23ff2
# ╟─af84d229-832a-48c8-ace2-defe1bf3b468
# ╟─608af765-55b4-4412-b293-513a25e35cd8
# ╠═179712a4-d8c9-419a-a708-15918d1dfa6e
# ╟─b940caf9-c0fc-43ea-b97d-0df5463bc1a6
# ╠═84fa53e7-f0f5-44c8-a976-b37f9356e9c0
# ╟─7e41f4e3-a2ae-4b39-aeaf-06c900b83135
# ╠═41e0de70-b3a4-43fc-ae35-1bf10a71c371
# ╟─40dc5eaf-fcc2-4803-a3d2-ce72126b77c9
# ╟─724d8110-5a41-477b-9079-6dee4987ecd4
# ╟─f665c520-8e26-4762-bf79-f01dada35a47
# ╠═deec9d4d-d701-46b2-b578-8a29623af096
# ╟─7ac6596a-d24b-4ae7-8339-7076cff64df0
# ╠═a0f39794-7bfb-42bf-8525-6824d97c36b1
# ╟─8a4a5e32-29c5-4d6e-88c9-82afa2ffd7b5
# ╟─fd954f51-857f-4a99-8029-0fe2b2723cda
# ╠═1b408007-5994-4f61-91ed-bb6f1a258114
# ╟─e2b31234-8ae3-492b-b61c-9d0bc38adec4
# ╟─9027b99d-2554-4341-b75a-8172932f239f
# ╟─81d68fa4-12dc-4183-8985-e7609febaf0c
# ╠═242c1317-df01-4859-bed5-dd2d3224cf50
# ╠═c67b3fb2-e9c7-49cc-a174-88c46c03030d
# ╠═c465abe9-50dc-4080-88d2-fae9a6191eff
# ╟─760d48c9-9d58-4b4c-9eee-d6df12aeec0f
# ╟─78230dbe-87a9-453b-ae23-97ff4ab0d03d
# ╠═e4f87f1a-44c1-4928-8d3c-af8f9a5fd2d5
# ╟─ad76082b-25bd-4a93-b590-2b5db3ea8245
# ╟─2c441c14-61da-43b5-ac1b-ee957769ce79
# ╠═8341dc4e-8fd7-4bb5-b1bb-bc31e9a71e92
# ╟─db2384ea-918c-447a-8a32-28106f753755
# ╠═5753000d-db74-4677-a074-ebd7c22692fd
# ╠═8e0077ce-1905-4c8e-ba27-0f3a734ab394
# ╟─cb0db38e-7124-4409-972a-a719012a09ad
# ╟─e199f8cc-b2de-4bd5-a077-41d683923cf6
# ╟─3d934675-e00c-4cc9-9073-f11baf52109d
# ╟─e574f5db-b7cf-4d4f-9803-026a43337f6d
# ╠═c3a70f7e-bf69-41f8-aecc-9c429d5de1c1
# ╠═9afbdf3c-9bdc-47aa-863a-8cee05013dba
# ╠═4624e243-952c-4549-bfc4-def2da8dfe4a
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002

max_stars_repo_path: notebooks/ML-Tutorial 2 - Using a Decision Tree.jl

max_stars_repo_name: roland-KA/JuliaForMLTutorial

max_issues_repo_name: roland-KA/JuliaForMLTutorial

max_forks_repo_name: roland-KA/JuliaForMLTutorial


text:
module HWconstrained

using JuMP, NLopt, DataFrames, Ipopt
using LinearAlgebra
export data, table_NLopt, table_JuMP

### Question 1
function data(a=0.5)
n=3
p=[1, 1, 1]
e=[2, 0, 0]
s1=s2=4
z1=[1, 1, 1, 1]
z2=[0.72, 0.92, 1.12, 1.32]
z3=[0.86, 0.96, 1.06, 1.16]
z=[[1, i, j] for i in z2 for j in z3]
z = vcat(z'...)
pi = repeat([1/16], 16)
a=0.5
na=3
nc=4
ns=4
nss=16

return Dict("a"=>a,"na"=>na,"nc"=>nc,"ns"=>ns,"nss"=>nss,"e"=>e,"p"=>p,"z"=>z,"pi"=>pi)
end
#end

d=data()

####  Question 2

function obj(x::Vector,grad::Vector,data::Dict)
    A = data["a"]
    Z = data["z"]
    pi = data["pi"]
    if length(grad) > 0
        grad[1] = A*exp.(-A*x[1])
        for i in 1:3
        grad[i+1] = sum(pi .* Z[:,i] .*A.*exp.(-A.*Z*x[i+1]))
        end
    end
    return -exp.(-A*x[1])+ sum(pi.*-exp.(-A*Z*x[2:4]))
end
obj(ones(4), zeros(4), d)

function constr(x::Vector,grad::Vector,data::Dict)
    if length(grad) > 0
        grad[1] = d["a"]*exp(-d["a"]*x[1])
        grad[2:end] = d["p"]
    end
    return x[1] + sum(d["p"].*(x[2:end].-d["e"]))
end

constr(ones(4), zeros(4), d) # keep track of # function evaluations

function max_NLopt(a=0.5)
d = data(a)
e= d["e"]
optimum = Opt(:LD_MMA, 4)
lower_bounds!(optimum, [0., -Inf, -Inf, -Inf])
max_objective!(optimum, (x, g)->obj(x, g, d))
inequality_constraint!(optimum, (x, g)->constr(x, g, d), 1e-8)
ftol_rel!(optimum, 1e-8)
NLopt.optimize(optimum, vcat(0,e))
end

function table_NLopt()
d = DataFrame(a=[0.5;1.0;5.0],c = zeros(3),omega1=zeros(3),omega2=zeros(3),omega3=zeros(3),fval=zeros(3))
for i in 1:nrow(d)
xx = max_NLopt(d[i,:a])
for j in 2:ncol(d)-1
d[i,j] = xx[2][j-1]
end
d[i,end] = xx[1]
end
return d
end

end # module

ext: jl


text:
############################################################################################
#                           CYLINDER CONSTRAINT          								   #
############################################################################################
"""
	CylinderConstraint{P,T}

Constraint of the form

	   NO constraint violation
-----------------------------------------
				NO constraint
				violation
				       r
	                 ---->
                 ////|////  ^
                 ////|////  |
  NO constraint  /// v ///  |     NO constraint
  violation      ////^////  |l    violation
                 ////|con.  |
                 ////|vio.  |
                 / p * ///  |

-----------------------------------------
	   NO constraint violation
The p is the origin, v is the direction, l is the length, r is the radius

# Constructor:
```julia
CylinderConstraint(n, p1::SVector{P}, p2::SVector{P}, p3::SVector{P},
	v::SVector{P},  l::SVector{P}, r::SVector{P}, x=1, y=2, z=3)
```
"""
struct CylinderConstraint{P,T} <: TrajectoryOptimization.StateConstraint
	n::Int
	p1::SVector{P,T}
	p2::SVector{P,T}
	p3::SVector{P,T}
	v::SVector{P,Symbol}
	l::SVector{P,T}
	r::SVector{P,T}
	x::Int  # index of x-state
	y::Int  # index of y-state
	z::Int  # index of z-state
	function CylinderConstraint{P,T}(n::Int,
		p1::AbstractVector, p2::AbstractVector, p3::AbstractVector,
		v::AbstractVector, l::AbstractVector, r::AbstractVector,
		x=1, y=2, z=3) where {P,T}
    	@assert length(p1) == length(p2) == length(p3) == length(v) == length(l) == length(r) == P "Lengths of p1, p2, p3, v, l, p must be equal. Got lengths ($(length(x1)), $(length(y1)), $(length(x2)), $(length(y2)), $(length(xv)), $(length(yv)))"
        new{P,T}(n, p1, p2, p3, v, l, r, x, y, z)
    end
end

function CylinderConstraint(n::Int,
		p1::AbstractVector, p2::AbstractVector, p3::AbstractVector,
		v::AbstractVector, l::AbstractVector, r::AbstractVector,
		x=1, y=2, z=3)
    T = promote_type(
		eltype(p1), eltype(p2), eltype(p3),
		eltype(l), eltype(r))
    P = length(p1)
    CylinderConstraint{P,T}(n, p1, p2, p3, v, l, r, x, y, z)
end

TrajectoryOptimization.state_dim(con::CylinderConstraint) = con.n

function TrajectoryOptimization.evaluate(con::CylinderConstraint, X::StaticVector)
	p1 = con.p1
	p2 = con.p2
	p3 = con.p3
	v = con.v
	l = con.l
	r = con.r
	x = X[con.x]
	y = X[con.y]
	z = X[con.z]

	# t0 = x - p
	t0_1 = x .- p1
	t0_2 = y .- p2
	t0_3 = z .- p3

	valid =
		((v .== :x) .& (t0_1 .> 0.0) .& (t0_1 .< l)) .|
		((v .== :y) .& (t0_2 .> 0.0) .& (t0_2 .< l)) .|
		((v .== :z) .& (t0_3 .> 0.0) .& (t0_3 .< l))

	out = r.^2 - t0_1.^2 - t0_2.^2 - t0_3.^2 +
		(v .== :x) .* t0_1.^2 +
		(v .== :y) .* t0_2.^2 +
		(v .== :z) .* t0_3.^2
	return out .* valid
end

function TrajectoryOptimization.jacobian!(∇c, con::CylinderConstraint{P}, X::SVector) where P
	p1 = con.p1
	p2 = con.p2
	p3 = con.p3
	v = con.v
	l = con.l
	r = con.r
	x = X[con.x]
	y = X[con.y]
	z = X[con.z]

	# t0 = x - p
	t0_1 = x .- p1
	t0_2 = y .- p2
	t0_3 = z .- p3

	valid =
		((v .== :x) .& (t0_1 .> 0.0) .& (t0_1 .< l)) .|
		((v .== :y) .& (t0_2 .> 0.0) .& (t0_2 .< l)) .|
		((v .== :z) .& (t0_3 .> 0.0) .& (t0_3 .< l))

	out = - r.^2 + t0_1.^2 + t0_2.^2 + t0_3.^2 -
		(v .== :x) .* t0_1.^2 -
		(v .== :y) .* t0_2.^2 -
		(v .== :z) .* t0_3.^2

	for i = 1:P
		∇c[i,con.x] = - valid[i] * 2*t0_1[i] * !(v[i] == :x)
		∇c[i,con.y] = - valid[i] * 2*t0_2[i] * !(v[i] == :y)
		∇c[i,con.z] = - valid[i] * 2*t0_3[i] * !(v[i] == :z)
	end
	return false
end

@inline Base.length(::CylinderConstraint{P}) where P = P
@inline TrajectoryOptimization.sense(::CylinderConstraint) = Inequality()

function TrajectoryOptimization.change_dimension(con::CylinderConstraint, n::Int, m::Int, ix=1:n, iu=1:m)
	CylinderConstraint(n,
		con.p1, con.p2, con.p3,
		con.v1, con.v2, con.v3,
		con.l, con.r,
		ix[con.x], ix[con.y], ix[con.z])
end

# # t0 = x - p
# t0_1 = x .- p1
# t0_2 = x .- p1
# t0_3 = x .- p1
# # t1 = t0 x v
# t1_1 = t0_2 .* v3 - t0_3 .* b2
# t1_2 = t0_3 .* v1 - t0_1 .* b3
# t1_3 = t0_1 .* v2 - t0_2 .* b1
# # t2 = norm(t1)
# t2 = sqrt.(t1_1.^2 + t1_2.^2 + t1_3.^2)
# far = t2 .> r
#
# out = (r .- t2).^2
# return out .* bottom .* top .* far

#
# tmp    = (x .- p1) .* v1 + (x .- p2) .* v2 + (x .- p3) .* v3
# top    = tmp .> l
# bottom = tmp .< 0.0
#
# # # t0 = x - p
# # t0_1 = x .- p1
# # t0_2 = y .- p2
# # t0_3 = z .- p3
# # # t1 = norm(t0)
# # t1 = sqrt.(t0_1.^2 + t0_2.^2 + t0_3.^2)
# # # t2 = t0'v
# # t2 = tmp
# # # t3 = t0 - t0'v t0/norm(t0)
# # t3_1 = t0_1 - t2 .* t0_1 ./ t1
# # t3_2 = t0_1 - t2 .* t0_2 ./ t1
# # t3_3 = t0_1 - t2 .* t0_3 ./ t1
# # # t4 = t3 x v
# # t4_1 = t3_2 .* v3 - t3_3 .* b2
# # t4_2 = t3_3 .* v1 - t3_1 .* b3
# # t4_3 = t3_1 .* v2 - t3_2 .* b1
# # # t5 = norm(t3)
# # t5 = sqrt.(t3_1.^2 + t3_2.^2 + t3_3.^2)
#
# far = t5 .> r
# out = (r .- t5).^2


text:
# -*- coding: utf-8 -*-
# ---
# jupyter:
#   jupytext:
#     formats: jl:light,ipynb
#     text_representation:
#       extension: .jl
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.5.1
#   kernelspec:
#     display_name: Julia 1.4.2
#     language: julia
#     name: julia-1.4
# ---

# ## DFT: GGA & Meta GGA Kernels

# ### Theory
#
#
# In density functional theory, we are interested in the energy expression:
#
# $$
# E_{xc} = D_{\mu \nu}^{T}(T_{\mu \nu}  + V_{\mu \nu}) + \frac{1}{2} D_{\mu \nu }^{T} D_{\lambda \sigma}^T (\mu \nu|\lambda \sigma) + E_{xc}[\rho_{\alpha}({\vec{r})}, \rho_{\beta}({\vec{r})}]
# $$
#
#
#
#
#
# Although the exchange correlation energy $E_{xc}$ is a functional of the density alone, the dependence on  $\rho_{\sigma}(\vec{r})$ is highly non-local. Because of this, small variations of the densities may cause large variations of the exchange correlation potential $v_{xc}$. Additionally, $v_{xc}$ at a given point $\vec{r}_i$ may be sensitive to changes at very distant points of $\vec{r}_j$.
#
#
#
# In order to overcome this, both semilocal and nonlocal ingredients must be added to the energy density. As expressed on the grid, these include:
#
# The gradient of the density:
# $$
# g = |\nabla \rho(\vec(r)) |
# $$
# The laplacian of the density:
# $$
# l = \nabla^2 \rho (\vec{r})
# $$
# And the non-interacting kinetic energy density:
# $$
# \tau = \frac{1}{2} \sum_k^{occ.} | \nabla \psi_k (\vec{r}) |^2
# $$
#
# Different functional approximations are defined by what ingredients are required to be created. Here, we concentrate on GGAs and meta-GGAs:
#
#
# $$
# E_{xc}^{GGA}[\rho] = \int f(\rho, g) \cdot d\vec{r}
# $$
# $$
# E_{xc}^{MGGA}[\rho] = \int f(\rho, g, l, \tau) \cdot d\vec{r}
# $$
#
# Where the integrands are known as the kernel, or the exchange-correlation energy density. It is clear here that, the more sophisticated density functional approximations, the more components are added to the exchange-correlation energy and potential.
#
#
# In practice we need to build a Kohn-Sham matrix:
#
# $$
# F_{\mu \nu}^{\alpha} = H_{\mu \nu} + J_{\mu \nu} + V_{\mu \nu}^{xc},
# $$
#
# where the last therm is the exchange-correlation contribution $V_{\mu \nu}^{xc}$ that is defined as the functional derivative of the energy with respect to the density.
#
#
# $$
# V^{xc} = \frac{\partial E_{xc}}{\partial D_{ab}}
# $$
#
# Once we have the $V^{xc}$ we can build the Kohn-Sham matrix and solve self consistently.
#

# +
using PyCall: pyimport
psi4 = pyimport("psi4")
np   = pyimport("numpy") # used only to cast to Julia Arrays

mol = psi4.geometry("""
He

symmetry c1
""")
psi4.set_options(Dict("BASIS" => "cc-pvdz",
                      "DFT_SPHERICAL_POINTS" => 6,
                      "DFT_RADIAL_POINTS" => 5))

psi4.core.set_output_file("output.dat", false)

gga_energy, gga_wfn = psi4.energy("PBE", return_wfn=true)
meta_energy, meta_wfn = psi4.energy("TPSS", return_wfn=true)

Vpot = gga_wfn.V_potential()
Vpot_meta = meta_wfn.V_potential()
# -

# ## Building the GGA kernel
#
#
# From the LDA tutorial, we have seen how to obtain the density from the basis functions:
#
# $$
# \rho_{\sigma}(\vec{r}) = D_{\mu \nu}^{\sigma} \phi_{\mu}(\vec{r}) \phi_{\nu}(\vec{r})
# $$
#
#
#
#
# The GGA depends on the gradient of the density. Using a basis set it is calculated as:
#
#
# $$
# \nabla_{\sigma} \rho{} (\vec{r}) = 2  D_{\mu \nu}^{\sigma}  \phi_{\mu}(\vec{r}) \nabla \phi_{\nu}(\vec{r})
# $$
#
# So that we can produce $\gamma$:
#
#
# $$
# \gamma_{\alpha \alpha}(\vec{r}) = \nabla \rho_{\alpha}(\vec{r}) \cdot \nabla \rho_{\alpha}(\vec{r})
# $$
#
#
# We then need to get the energy from the kernel by doing a numerical integration:
# $$e_{\rm{xc}} = w^a f^a_{\rm{xc}}$$
#
# Where the $w_{\alpha}$ correspond to the combined Truetler and Lebedev weights at each point needed for the numerical quadrature.
#
# Finally, the potential on the grid will be given by the derivative of the kernel with respect to gamma:
#
# $$
# V^{\gamma} = 2 \frac{\partial f}{\partial \gamma_{\alpha\alpha}} \nabla \rho_{\alpha} + \frac{\partial f}{\partial \gamma_{\alpha\beta}} \nabla \rho_{\beta}
# $$
#
#
# And it can be added and then it needs to get reintegrated back to atomic orbital space:
#
# $$
# V_{ab}^{\gamma} = \int_{\mathbb{R}^3} V^{\gamma} \nabla (\phi_{\mu} \phi_{\nu}) d\vec{r}
# $$
#
# The next calculation assumes that the density matrix $D$ is symmetric. This means that $ \nabla \phi(\vec{r}) D_{\mu \nu} \phi(\vec{r})= \phi(\vec{r}) D_{\mu \nu} \nabla \phi(\vec{r})$.
# One then ought to be careful with systems where this condition is not met, for example CPHF.
#
# Specifically,  here write this matrix contribution as:
#
# $$
# V_{ab}^{\gamma} = 4 \cdot \nabla \phi_{\mu}(\vec{r}) \cdot V_{\alpha \alpha}^{\gamma} \cdot \nabla n_{p}(\vec{r}) \cdot w_{\alpha} \cdot \phi_{\nu} (\vec{r})
# $$
#
#

# +
#GGA Kernel

D = np.array(gga_wfn.Da())

V = zero(D)
xc_e = 0.0

rho = []
points_func = Vpot.properties()[1]
superfunc = Vpot.functional()

xc_e, V = let xc_e = xc_e, rho=rho

   # Loop over the blocks
   for b in 1:Vpot.nblocks()

       # Obtain block information
       block = Vpot.get_block(b-1)
       points_func.compute_points(block)
       npoints = block.npoints()
       lpos = np.array(block.functions_local_to_global()) .+ 1

       # Obtain the grid weight
       w = np.array(block.w())

       # Compute phi!
       phi = np.array(points_func.basis_values()["PHI"])[1:npoints, 1:size(lpos,1)]

       phi_x = np.array(points_func.basis_values()["PHI_X"])[1:npoints, 1:size(lpos,1)]
       phi_y = np.array(points_func.basis_values()["PHI_Y"])[1:npoints, 1:size(lpos,1)]
       phi_z = np.array(points_func.basis_values()["PHI_Z"])[1:npoints, 1:size(lpos,1)]

       # Build a local slice of D
       lD = D[lpos, lpos]

       # Compute rho
       # ρ[p] = ϕ[pμ] lD[μν] ϕ[pν]
       rho = 2vec(sum( (phi * lD) .* phi, dims=2))

       # 2.0 for Px D P + P D Px, 2.0 for non-spin Density
       # ∇ϕₓ[p] = 4 ϕ[pμ] lD[μν] ∇ₓϕ[pν] ∀ x,y,z
       rho_x = 4vec(sum( (phi * lD) .* phi_x, dims=2))
       rho_y = 4vec(sum( (phi * lD) .* phi_y, dims=2))
       rho_z = 4vec(sum( (phi * lD) .* phi_z, dims=2))
       gamma = @. rho_x^2 + rho_y^2 + rho_z^2

       inp = Dict()
       inp["RHO_A"] = psi4.core.Vector.from_array(rho)
       inp["GAMMA_AA"] = psi4.core.Vector.from_array(gamma)

       # Compute the kernel
       ret = superfunc.compute_functional(inp, -1)

       # Compute the XC energy
       vk = np.array(ret["V"])[1:npoints]
       xc_e += sum(w .* vk)

       # Compute the XC derivative.
       v_rho_a = np.array(ret["V_RHO_A"])[1:npoints]
       # Vtmp[ab] = 0.5 ϕ[pb] Vρ_a[p] w[p] ϕ[pa]
       Vtmp = 0.5phi' * (v_rho_a .* w .* phi)

       #Comute gamma and its associated potential
       v_gamma_aa = np.array(ret["V_GAMMA_AA"])[1:npoints]
       # Vtmp[ab] += 2 ∇ₓϕ[pb] Vγ_aa[p] ∇ₓρ[p] w[p] ϕ[pa] ∀ x,y,z
       Vtmp += 2phi_x' * (v_gamma_aa .* rho_x .* w .* phi)
       Vtmp += 2phi_y' * (v_gamma_aa .* rho_y .* w .* phi)
       Vtmp += 2phi_z' * (v_gamma_aa .* rho_z .* w .* phi)

       # Add the temporary back to the larger array by indexing, ensure it is symmetric
       V[lpos, lpos] += (Vtmp + Vtmp')
   end
   xc_e, V
end

println("XC Energy ", xc_e)
println("V matrix:")
display(V)

println("\nMatches Psi4 V: ", np.allclose(V, gga_wfn.Va()))

# -

# ## Building the meta-GGA kernel
#
#
# Just like we did with GGA, meta-GGA requires an extra component to be added. In this case is the kinetic energy density.
#
# $$
# \tau_{\sigma} (\vec{r}) = D_{\mu \nu}^{\sigma} \nabla \phi_{\mu}(\vec{r}) \nabla \phi_{\nu}(\vec{r})
# $$
#
# We calculate the $E_{xc}$ again with $ w^a f^a_{\rm{xc}}$.
#
# And finally, the $\tau$ potential contribution can be calculated as:
#
# $$
# V^{\tau} = \frac{\partial f}{\partial \tau}
# $$
#
# Which is expressed as a matrix in the basis sest as:
#
# $$
# V_{\mu \nu}^{\tau} = \int_{\mathbb{R}^3} V^{\tau} \nabla \phi_{\mu} \nabla \phi_{\nu} d\vec{r}
# $$
#
#
# In the code we calculate this contribution like so:
#
# $$
# V_{\mu \nu}^{\tau} = \frac{1}{2}  \cdot \nabla \phi_{\mu} \cdot V^{\tau}_{a} \cdot w_{a} \cdot \nabla \phi_{\nu}
# $$
#
#
#

# +
#meta-GGA Kernel

D = np.array(meta_wfn.Da())

V = zero(D)
xc_e = 0.0

rho = []
points_func = Vpot_meta.properties()[1]
superfunc = Vpot_meta.functional()

xc_e, V = let xc_e = xc_e, rho=rho

   # Loop over the blocks
   for b in 1:Vpot.nblocks()

       # Obtain block information
       block = Vpot.get_block(b-1)
       points_func.compute_points(block)
       npoints = block.npoints()
       lpos = np.array(block.functions_local_to_global()) .+ 1

       tau = np.zeros(npoints)

       # Obtain the grid weight
       w = np.array(block.w())

       # Compute phi!
       phi = np.array(points_func.basis_values()["PHI"])[1:npoints, 1:size(lpos,1)]

       phi_x = np.array(points_func.basis_values()["PHI_X"])[1:npoints, 1:size(lpos,1)]
       phi_y = np.array(points_func.basis_values()["PHI_Y"])[1:npoints, 1:size(lpos,1)]
       phi_z = np.array(points_func.basis_values()["PHI_Z"])[1:npoints, 1:size(lpos,1)]

       # Build a local slice of D
       lD = D[lpos, lpos]

       # Compute rho
       # ρ[p] = 2 ϕ[pm] lD[mn] ϕ[pn]
       rho = 2vec(sum( (phi * lD) .* phi, dims=2))

       # 2.0 for Px D P + P D Px, 2.0 for non-spin Density
       # ∇ₓρ[p] = 4 ϕ[pm] lD[mn] ∇ₓϕ[pn]
       rho_x = 4vec(sum( (phi * lD) .* phi_x, dims=2))
       rho_y = 4vec(sum( (phi * lD) .* phi_y, dims=2))
       rho_z = 4vec(sum( (phi * lD) .* phi_z, dims=2))
       gamma = @. rho_x^2 + rho_y^2 + rho_z^2

       #Compute Tau
       # τ[p] = ∇ₓϕ[pm] lD[mn] ∇ₓϕ[pn]
       tau  = vec(sum( (phi_x * lD) .* phi_x, dims=2))
       tau += vec(sum( (phi_y * lD) .* phi_y, dims=2))
       tau += vec(sum( (phi_z * lD) .* phi_z, dims=2))

       inp = Dict()
       inp["RHO_A"] = psi4.core.Vector.from_array(rho)
       inp["GAMMA_AA"] = psi4.core.Vector.from_array(gamma)
       inp["TAU_A"]= psi4.core.Vector.from_array(tau)

       # Compute the kernel
       ret = superfunc.compute_functional(inp, -1)

       # Compute the XC energy
       vk = np.array(ret["V"])[1:npoints]
       xc_e += sum(w .* vk)

       # Compute the XC derivative.
       v_rho_a = np.array(ret["V_RHO_A"])[1:npoints]
       # Vtmp[ab] = 0.5 ϕ[pb] Vρ_a[p] w[p] ϕ[pa]
       Vtmp = 0.5phi' * (v_rho_a .* w .* phi)

       #Compute gamma and its potential matrix
       v_gamma_aa = np.array(ret["V_GAMMA_AA"])[1:npoints]
       # Vtmp[ab] = 2∇ₓϕ[pb] Vγ_aa[p] ∇ₓϕ[p] w[p] ϕ[pa] ∀ x,y,z
       Vtmp += 2phi_x' * (v_gamma_aa .* rho_x .* w .* phi)
       Vtmp += 2phi_y' * (v_gamma_aa .* rho_y .* w .* phi)
       Vtmp += 2phi_z' * (v_gamma_aa .* rho_z .* w .* phi)

       #Compute Vτ
       v_tau_a = np.array(ret["V_TAU_A"])[1:npoints]
       # Vtmp[ab] += 0.5 ∇ₓϕ[pb] Vτ_a[p] w[p] ∇ₓϕ[pa] ∀ x,y,z
       Vtmp += 0.5phi_x' * (v_tau_a .* w .* phi_x)
       Vtmp += 0.5phi_y' * (v_tau_a .* w .* phi_y)
       Vtmp += 0.5phi_z' * (v_tau_a .* w .* phi_z)

       # Add the temporary back to the larger array by indexing, ensure it is symmetric
       V[lpos, lpos] += (Vtmp + Vtmp')
   end
   xc_e, V
end

println("XC Energy ", xc_e)
println("V matrix:")
display(V)

println("\nMatches Psi4 V: ", np.allclose(V, meta_wfn.Va()))

# -

# #### To put all the approximations into perspective, let us look at every component of the meta-GGA exchange-correlation potential that we just created.
#
# $$
# V_{\mu \nu}^{xc, \alpha} = \int_{\mathbb{R}^3} \bigg( \frac{\partial f}{\partial \rho_{\alpha}} \bigg)  \phi_{\mu} \phi_{\nu} d\vec{r}
# $$
#
#
#
#
#

# $$
# +\int_{\mathbb{R}^3} \bigg(2 \frac{\partial f}{\partial \gamma_{\alpha\alpha}} \nabla \rho_{\alpha} + \frac{\partial f}{\partial \gamma_{\alpha\beta}} \bigg)  \nabla \rho_{\beta} \nabla (\phi_{\mu} \phi_{\nu}) d\vec{r}
# $$
#

# $$
# \int_{\mathbb{R}^3} \bigg( \frac{\partial f}{\partial \tau} \bigg)  \nabla \phi_{\mu} \nabla \phi_{\nu} d\vec{r}
# $$
#
# Here every line represent each of the rungs in the systematic methodology of density functional aproximations, the first line corresponds to LDA, addition of the second line corresponds to GGA and addition to the third line corresponds to meta-GGA.

# ## References
#
# 1. Original papers:
# 	> [[Hohenberg:1964:136](https://journals.aps.org/pr/abstract/10.1103/PhysRev.136.B864)] P. Hohenberg and W. Kohn, Phys. Rev. 136, B864-B871, **1964**.
#
#     > [[Kohn:1965:A1133](https://journals.aps.org/pr/abstract/10.1103/PhysRev.140.A1133)] W. Kohn and L.J. Sham, Phys. Rev. 140, A1133-A1138, **1965**.
# 2. Analytic derivatives and algorithm:
#     > [[Johnson:1994:100](https://aip.scitation.org/doi/abs/10.1063/1.466887)] Johnson, B. G.; Fisch M. J.; *J. Chem. Phys.*, **1994**, *100*, 7429
# 4. Additional information:
# 	> [[Staroverov:2012](https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118431740#page=156)] Staroverov, Viktor N. "Density-functional approximations for exchange and correlation." A Matter of Density, **2012**: 125-156.
#
#     > [[Parr:1989](https://link.springer.com/chapter/10.1007/978-94-009-9027-2_2)] R.G. Parr and W. Yang, Density Functional Theory of Atoms and Molecules Oxford University Press, USA, 1989 ISBN:0195357736, 9780195357738


text:
using OpenStreetMapParser, FactCheck

MAP_FILENAME = "tech_square.osm"

if !isfile(MAP_FILENAME)
    url = "https://dl.dropboxusercontent.com/u/8297575/$MAP_FILENAME"
    download(url, MAP_FILENAME)
end

nodes = parseNodes(MAP_FILENAME)
@fact length(nodes.ids) => length(nodes.latlon)
@fact length(nodes.ids) => 1410

@fact length(nodes.tags) => 24
@fact sort(collect(keys(nodes.tags))) =>
    UTF8String["FIXME", "addr:housenumber", "addr:postcode", "addr:street",
               "amenity", "atm", "attribution", "bus", "created_by", "crossing",
               "highway", "line", "massgis:geom_id", "name", "network", "noexit",
               "operator", "public_transport", "railway", "ref", "source",
               "traffic_signals:sound", "url", "wikipedia"]

crossing = nodes.tags["crossing"]
@fact length(crossing) => 14
@fact crossing[61326778]   => "traffic_signals"
@fact crossing[1628602211] => "traffic_signals"
@fact crossing[1628602029] => "traffic_signals"
@fact crossing[1705312831] => "zebra"
@fact crossing[1705308323] => "zebra"
@fact crossing[61317367]   => "traffic_signals"
@fact crossing[1628602239] => "traffic_signals"
@fact crossing[1628602041] => "traffic_signals"
@fact crossing[1628602219] => "traffic_signals"
@fact crossing[1628602201] => "traffic_signals"
@fact crossing[61329208]   => "traffic_signals"
@fact crossing[1628602010] => "traffic_signals"
@fact crossing[1705308324] => "zebra"
@fact crossing[61327535]   => "traffic_signals"

noexit = nodes.tags["noexit"]
@fact length(noexit) => 1
@fact noexit[1230207187] => "yes"

line = nodes.tags["line"]
@fact length(line) => 1
@fact line[69480325] => "Red"

railway = nodes.tags["railway"]
@fact length(railway) => 6
railway[1819633311] => "level_crossing"
railway[69480325]   => "station"
railway[365930364]  => "level_crossing"
railway[1819633315] => "level_crossing"
railway[1819633313] => "level_crossing"
railway[1819633310] => "level_crossing"

highway = nodes.tags["highway"]
@fact highway[1034918607] => "crossing"
@fact highway[1053454370] => "traffic_signals"
@fact highway[1052804034] => "traffic_signals"
@fact highway[61318536]   => "traffic_signals"
@fact highway[1829969262] => "crossing"
@fact highway[1053524037] => "traffic_signals"
@fact highway[1829969266] => "crossing"
@fact highway[1052777483] => "traffic_signals"
@fact highway[1829975813] => "crossing"
@fact highway[1053454390] => "crossing"
@fact highway[1068396444] => "crossing"
@fact highway[1829975825] => "crossing"
@fact highway[1052761909] => "traffic_signals"
@fact highway[1053524038] => "crossing"
@fact highway[2291907804] => "crossing"
@fact highway[61321277]   => "traffic_signals"
@fact highway[1068396443] => "crossing"
@fact highway[1829969270] => "crossing"
@fact highway[1705299287] => "turning_circle"
@fact highway[1053508679] => "traffic_signals"

amenity = nodes.tags["amenity"]
@fact length(amenity) => 4
@fact amenity[1710142648] => "bank"
@fact amenity[1914343564] => "restaurant"
@fact amenity[1749306041] => "restaurant"
@fact amenity[182895807]  => "university"

addr = nodes.tags["addr:housenumber"]
@fact length(addr) => 1
@fact addr[1710142648] => "700"

name = nodes.tags["name"]
@fact length(name) => 8
name[1399777651] => "Broadway opp Hampshire St"
name[1710142648] => "MIT Federal Credit Union"
name[1914343564] => "Catalyst"
name[69480325]   => "Alewife"
name[1399777278] => "Broadway @ Galileo Way"
name[1749306041] => "Area Four"
name[182895807] => "McGovern Institute for Brain Research (MIT)"
name[1399777296] => "Broadway @ Hampshire St"

atm = nodes.tags["atm"]
@fact length(atm) => 1
@fact atm[1710142648] => "yes"

@fact nodes.tags["addr:street"] => Dict(1710142648=>"Technology Square")
@fact nodes.tags["addr:postcode"] => Dict(1710142648=>"02139")
@fact nodes.tags["created_by"] => Dict(61327003 => "JOSM", 61327001 => "JOSM",
                                       61326996 => "JOSM", 61326990 => "JOSM",
                                       61326983 => "JOSM", 61326992 => "JOSM",
                                       61326994 => "JOSM", 61327000 => "JOSM",
                                       61317286 => "JOSM", 61332582 => "JOSM")
@fact nodes.tags["ref"] => Dict(1399777651 => "24485", 1399777278 => "2228",
                                1399777296 => "24486")

ways = parseWays(MAP_FILENAME)
@fact length(ways.ids) => 110
@fact length(ways.ids) => length(ways.nodes)
@fact length(ways.tags) => 89
@fact ways.tags["created_by"] => Dict(8615138  => "JOSM",
                                      29860015 => "Potlatch 0.10f",
                                      8616240  => "Potlatch 0.9c")
@fact ways.tags["name:he"] => Dict(27366257=>"המכון הטכנולוגי של מסצ'וסטס")
@fact ways.tags["massgis:DCAM_ID"] => Dict(29835182=>"0")
@fact ways.tags["layer"] => Dict(236626982=>"-1")
@fact ways.tags["gnis:state_id"] => Dict(27366257=>"25")
@fact ways.tags["ref"] => Dict(29740162 => "NE49", 17660188 => "46",
                               29558550 => "NE80", 29609886 => "NE47",
                               29650626 => "NE48", 29803755 => "48")
@fact ways.tags["massgis:OS_DEED_PA"] => Dict(29835182=>"0")
@fact ways.tags["ramp:wheelchair"] => Dict(106991439=>"yes")
@fact ways.tags["massgis:TOWN_ID"] => Dict(29835182=>"49")
@fact length(ways.tags["building"]) => 32
@fact ways.tags["massgis:ref"] => Dict(90719013  => "0 N324", 239457938 => "0 N323",
                                       8614848   => "0 N324", 33720430  => "0 N324",
                                       34065677  => "0 N323", 239457937 => "0 N323",
                                       138749269 => "0 N323", 8614999   => "0 N324")

relations = parseRelations(MAP_FILENAME)

@fact relations.ids => [1763392, 2119819, 2119824, 2119848, 1600292, 1412944]

@fact length(relations.members) => 6
@fact relations.members[2119819] => [Dict("role"=>"outer","type"=>"way","ref"=>"158302371"),
                                     Dict("role"=>"inner","type"=>"way","ref"=>"29860629"),
                                     Dict("role"=>"inner","type"=>"way","ref"=>"29624640")]

@fact relations.members[2119824] => [Dict("role"=>"outer","type"=>"way","ref"=>"38096750"),
                                     Dict("role"=>"inner","type"=>"way","ref"=>"29947888"),
                                     Dict("role"=>"inner","type"=>"way","ref"=>"29810320"),
                                     Dict("role"=>"inner","type"=>"way","ref"=>"29863742"),
                                     Dict("role"=>"inner","type"=>"way","ref"=>"158302371")]
@fact relations.members[2119848] => [Dict("role"=>"outer","type"=>"way","ref"=>"158305166"),
                                    Dict("role"=>"inner","type"=>"way","ref"=>"158305165"),
                                    Dict("role"=>"inner","type"=>"way","ref"=>"158305356")]
@fact relations.members[1763392] => [Dict("role"=>"via","type"=>"node","ref"=>"1034918626"),
                                     Dict("role"=>"from","type"=>"way","ref"=>"8615585"),
                                     Dict("role"=>"to","type"=>"way","ref"=>"233319829")]
@fact length(relations.members[1600292]) => 58
@fact length(relations.members[1412944]) => 18

@fact length(relations.tags) => 17
@fact relations.tags["name"] => Dict(2119824 => "One Kendall Square",
                                     1600292 => "Red Line",
                                     1412944 => "Grand Junction Branch")
@fact relations.tags["wikipedia"] => Dict(1600292=>"en:Red Line (MBTA)")
@fact relations.tags["old_railway_operator"] => Dict(1412944=>"NYC")
@fact relations.tags["ref"] => Dict(1600292=>"Red Line")
@fact relations.tags["landuse"] => Dict(2119824=>"commercial")
@fact relations.tags["colour"] => Dict(1600292=>"#FF0000")
@fact relations.tags["route"] => Dict(1600292=>"subway",1412944=>"railway")
@fact relations.tags["area"] => Dict(2119819=>"yes",2119848=>"yes")
@fact relations.tags["highway"] => Dict(2119819=>"pedestrian",2119848=>"pedestrian")
@fact relations.tags["operator"] => Dict(1600292=>"MTBA")
@fact relations.tags["restriction"] => Dict(1763392=>"no_right_turn")
@fact relations.tags["network"] => Dict(1600292=>"BOS subway")
@fact relations.tags["website"] => Dict(2119824=>"http://www.onekendallsquare.com/")
@fact relations.tags["to"] => Dict(1600292=>"Mattapan")
@fact relations.tags["from"] => Dict(1600292=>"Alewife")
@fact relations.tags["type"] => Dict(2119819 => "multipolygon",
                                     2119824 => "multipolygon",
                                     2119848 => "multipolygon",
                                     1763392 => "restriction",
                                     1600292 => "route",
                                     1412944 => "route")
@fact relations.tags["FIXME"] => Dict(1412944=>"Was this originally continuous?")

nodes_df = osmDataFrame(nodes)
ways_df = osmDataFrame(ways)
relations_df = osmDataFrame(relations)

building = ways.tags["building"]
@fact length(building) => 32
@fact building[29842810]  => "yes"
@fact building[29740162]  => "university"
@fact building[29810276]  => "yes"
@fact building[37095599]  => "yes"
@fact building[212292730] => "yes"
@fact building[29866161]  => "yes"
@fact building[29627050]  => "yes"
@fact building[29625886]  => "yes"
@fact building[29627492]  => "yes"
@fact building[29628336]  => "yes"
@fact building[29947888]  => "yes"
@fact building[29575544]  => "yes"
@fact building[29558550]  => "university"
@fact building[29609886]  => "university"
@fact building[29862486]  => "yes"
@fact building[29650626]  => "university"
@fact building[29616645]  => "yes"
@fact building[29650632]  => "yes"

highways = ways.tags["highway"]
@fact length(highway) => 55
@fact highway[84967434]  => "footway"
@fact highway[106989193] => "footway"
@fact highway[158302370] => "service"
@fact highway[8615863]   => "footway"
@fact highway[34032974]  => "cycleway"
@fact highway[106987173] => "footway"
@fact highway[39551418]  => "footway"
@fact highway[239457940] => "secondary"
@fact highway[90758903]  => "secondary"
@fact highway[58336059]  => "service"
@fact highway[158304546] => "service"
@fact highway[106989189] => "pedestrian"
@fact highway[8614893]   => "secondary"
@fact highway[239457935] => "secondary"
@fact highway[8616187]   => "residential"
@fact highway[239457936] => "secondary"
@fact highway[149924230] => "secondary"
@fact highway[8615495]   => "secondary"
@fact highway[90719013]  => "secondary"
@fact highway[239457939] => "secondary"


text:
macro dprint(s)
    :(verbose && println($s))
end

function parsefromargs()
  s = ArgParseSettings("Process DCE-MRI data. "*
                       "Optional arguments can be used to override any "*
                       "values found in input files. "*
                       "For questions, contact David Smith <david.smith@gmail.com>. "*
                       "For bug reports and feature requests, file an issue at "*
                       "http://github.com/davidssmith/DCEMRI.jl"
                       )
  @add_arg_table s begin
    "datafile"
    help = "path to MAT file containing DCE and T1 data"
    arg_type = ByteString
    #required = true
    default = "input.mat"
    "--outfile", "-O"
    help = "path to MAT file to contain the ouput"
    arg_type = ByteString
    default = "results.mat"
    "--relaxivity", "-R"
    help = "contrast agent relaxivity (1/s)"
    arg_type = Float64
    "--TR", "-r"
    help = "repetition time (ms)"
    arg_type = Float64
    "--DCEflip", "-d"
    help = "flip angle of DCE data"
    arg_type = Float64
    "--SERcutoff", "-c"
    help = "minimum SER to include in processing mask"
    arg_type = Float64
    "--T1flip", "-t"
    help = "list of flip angle(s) of T1 data"
    arg_type = Float64
    nargs = '+'
    "--models", "-m"
    help = "list of models: 1=plasma only, 2=Standard, 3=Extended"
    arg_type = Int64
    nargs = '+'
    "--plotting", "-p"
    help = "plot intermediate results"
    action = :store_true
    "--workers", "-w"
    help = "number of parallel workers to use (one per CPU core is good)"
    arg_type = Int64
    default = 4
    "--verbose", "-v"
    help = "show verbose output"
    action = :store_true
  end

  parsed_args = parse_args(ARGS, s)

  if parsed_args["verbose"]
    println("Parsed args:")
    for (key,val) in parsed_args
      println("  $key  =>  $(repr(val))")
    end
  end
  parsed_args
end

function defaults()
  # defaults to use if not specified
  opts = Dict()
  opts["datafile"]   = "input.mat"
  opts["outfile"]    = "output.mat"
  opts["relaxivity"] = 4.5
  opts["SERcutoff"]  = 2.0
  opts["models"]     = [2]
  opts["plotting"]   = false
  opts["verbose"]    = true
  opts["workers"]    = 4
  opts
end

function startworkers(n)
  nadd = nworkers() == 1 ? n : n - nworkers()
  if nadd > 0
     println("adding $nadd workers")
     addprocs(nadd)
  end
end

function validate(d::Dict)
  for k in ["Cp", "DCEdata", "DCEflip", "TR", "t"]
    @assert haskey(d, k) "Input must contain $k"
  end
  @assert (haskey(d,"R10") && haskey(d,"S0")) || (haskey(d,"T1data") && haskey(d,"T1flip")) "Input must contain either R10+S0 or T1data+T1flip."
end

function ccc{T,N}(x::Array{T,N}, y::Array{T,N})
  # concordance correlation coefficient
  m1 = mean(x)
  m2 = mean(y)
  s1 = var(x)*(length(x) - 1.0) / length(x)
  s2 = var(y)*(length(y) - 1.0) / length(y)
  s12 = sum((x - m1).*(y - m2)) / length(x)
  2s12 / (s1 + s2 + (m1 - m2).^2)
end

# converts keyword argument to a dictionary
function kwargs2dict(kwargs)
    d = Dict{ASCIIString,Any}()
    for (k, v) in kwargs
        d[string(k)] = v
    end
    d
end