Scikit.ML.DataFrame
0.3.0.1
See the version list below for details.
dotnet add package Scikit.ML.DataFrame --version 0.3.0.1
NuGet\Install-Package Scikit.ML.DataFrame -Version 0.3.0.1
<PackageReference Include="Scikit.ML.DataFrame" Version="0.3.0.1" />
paket add Scikit.ML.DataFrame --version 0.3.0.1
#r "nuget: Scikit.ML.DataFrame, 0.3.0.1"
// Install Scikit.ML.DataFrame as a Cake Addin
#addin nuget:?package=Scikit.ML.DataFrame&version=0.3.0.1
// Install Scikit.ML.DataFrame as a Cake Tool
#tool nuget:?package=Scikit.ML.DataFrame&version=0.3.0.1
DataFrame for ML.net
This library provides an easy way to manipulate data
with ML.net.
It implements a subsample of pandas's dataframes API.
It only represents dense datasets but provides usual functionalities such as
expressions with []
and joins, group by, or sort functionalities.
Many examples can be found in
TestDataManipulation.cs.
Below, the following examples shows how to interact
with ML.net.
Changes
Version numbers refers to ML.net version numbers.
Version 0.3.0.1 (10/12/2018)
Fixes Issue #1, implements missing schema methods used to dump the output of a pipeline on disk.
Version 0.3.0 (10/7/2018)
Initial version, contains basic functionality of pandas API except pivots (so [], join, groupby, filter, sort).
Documentation
The dataframe contains a set of typed columns. All values in one column
must have the same type. Usually operators []
follows
pandas API
as well as with also loc
and iloc
.
Method Join
, GroupBy
, Sort
with at most three columns.
This limitations can easily be moved but cannot be infinite.
Whenever possible, the methods return a view on the original DataFrame,
a view is subset of rows and columns. The view is not necessarily smaller
than the original data.
Example 1: inner API
This example relies on the inner API, mostly used inside components of ML.net.
var env = new TlcEnvironment();
var iris = "iris.txt";
// We read the text data and create a dataframe / dataview.
var df = DataFrame.ReadCsv(iris, sep: '\t',
dtypes: new DataKind?[] { DataKind.R4 });
// We add a transform to concatenate two features in one vector columns.
var conc = env.CreateTransform("Concat{col=Feature:Sepal_length,Sepal_width}", df);
// We create training data by mapping roles to columns.
var trainingData = env.CreateExamples(conc, "Feature", label: "Label");
// We create a trainer, here a One Versus Rest with a logistic regression as inner model.
var trainer = env.CreateTrainer("ova{p=lr}");
using (var ch = env.Start("test"))
{
// We train the model.
var pred = trainer.Train(env, ch, trainingData);
// We compute the prediction (here with the same training data but it should not be the same).
var scorer = trainer.GetScorer(pred, trainingData, env, null);
// We store the predictions on a file.
DataFrame.ViewToCsv(env, scorer, "iris_predictions.txt");
// Or we could put the predictions into a dataframe.
var dfout = DataFrame.ReadView(scorer);
// And access one value...
var v = dfout.iloc[0, 7];
Console.WriteLine("PredictedLabel: {0}", v);
}
The current interface of DataFrame is not rich. It will improve in the future.
Example 2: EntryPoints API
This is the same example based on
Iris Classification
but using the new class DataFrame. It is not necessary anymore
to create a class specific to the data used to train. It is a
little bit less efficient for predictions as two consecutive
calls to method Predict
on generic data requires
some the pipeline to build new iterators at every call.
This extra work can be saved when the prediction instance is known.
var iris = "iris.txt";
// We read the text data and create a dataframe / dataview.
var df = DataFrame.ReadCsv(iris, sep: '\t',
dtypes: new DataKind?[] { DataKind.R4 });
var importData = df.EPTextLoader(iris, sep: '\t', header: true);
var learningPipeline = new GenericLearningPipeline();
learningPipeline.Add(importData);
learningPipeline.Add(new ColumnConcatenator("Features", "Sepal_length", "Sepal_width"));
learningPipeline.Add(new StochasticDualCoordinateAscentRegressor());
var predictor = learningPipeline.Train();
var predictions = predictor.Predict(df);
var dfout = DataFrame.ReadView(predictions);
// And access one value...
var v = dfout.iloc[0, 7];
Console.WriteLine("{0}: {1}", vdf.Schema.GetColumnName(7), v.iloc[0, 7]);
Example 3: DataFrame in C#
The class DataFrame
replicates some functionalities
datascientist are used to in others languages such as
Python or R. It is possible to do basic operations
on columns:
var text = "AA,BB,CC\n0,1,text\n1,1.1,text2";
var df = DataFrame.ReadStr(text);
df["AA+BB"] = df["AA"] + df["BB"];
Console.WriteLine(df.ToString());
AA,BB,CC,AA+BB
0,1,text,1
1,1.1,text2,2.1
Or:
df["AA2"] = df["AA"] + 10;
Console.WriteLine(df.ToString());
AA,BB,CC,AA+BB,AA2
0,1,text,1,10
1,1.1,text2,2.1,11
The next instructions change one value based on a condition.
df.loc[df["AA"].Filter<DvInt4>(c => (int)c == 1), "CC"] = "changed";
Console.WriteLine(df.ToString());
AA,BB,CC,AA+BB,AA2
0,1,text,1,10
1,1.1,changed,2.1,11
A specific set of columns or rows can be extracted:
var view = df[df.ALL, new [] {"AA", "CC"}];
Console.WriteLine(view.ToString());
AA,CC
0,text
1,changed
The dataframe also allows basic filtering:
var view = df[df["AA"] == 0];
Console.WriteLine(view.ToString());
AA,BB,CC,AA+BB,AA2
0,1,text,1,10
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
.NET Standard | netstandard2.0 is compatible. netstandard2.1 was computed. |
.NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
MonoAndroid | monoandroid was computed. |
MonoMac | monomac was computed. |
MonoTouch | monotouch was computed. |
Tizen | tizen40 was computed. tizen60 was computed. |
Xamarin.iOS | xamarinios was computed. |
Xamarin.Mac | xamarinmac was computed. |
Xamarin.TVOS | xamarintvos was computed. |
Xamarin.WatchOS | xamarinwatchos was computed. |
-
.NETStandard 2.0
- Microsoft.ML (>= 0.3.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.