Regression - Instruction

Andreas Bauer · Jun 8, 2010

Hello,

a b c
fx 0,639242 0,032636 19,819060
fy 0,003413 -0,646874 273,495500

MustBe x MustBe y Measure X Measure Y
173,000 12,500 130,816 266,000
275,500 28,500 196,860 256,000
139,500 153,000 113,987 175,000

a) How can I calculate it with C#?
b) How is the correct algorithm? If I make it by hand.
Have somebody a good instruction?
c) Maybe I can use Excel, or?

It is Taylor?

Thanks. Regards Andreas

Arne Vajhøj · Jun 9, 2010

a b c
fx 0,639242 0,032636 19,819060
fy 0,003413 -0,646874 273,495500

MustBe x MustBe y Measure X Measure Y
173,000 12,500 130,816 266,000
275,500 28,500 196,860 256,000
139,500 153,000 113,987 175,000

a) How can I calculate it with C#?
b) How is the correct algorithm? If I make it by hand.
Have somebody a good instruction?
c) Maybe I can use Excel, or?

It is Taylor?

Maybe you should start by explaining in more detail
what this is.

It is not obvious what the data represents nor what you
want to calculate.

Arne

Andreas Bauer · Jun 10, 2010

Hello Giovanni,
here I send more details and hope is clear.
I need more informations, it is not so easy.

I hope the pictures are clear.
http://www1.minpic.de/bild_anzeigen.php?id=114414&key=77476860&ende

http://www1.minpic.de/bild_anzeigen.php?id=114415&key=92154987&ende

First, I measure my 3 or more points.
Second, I check the units(steps) to the axis. (Step motor)
Third, I should calculate the coefficient.

How is the best, easiest way?

Thanks for tipps.

--Simple linear regression, or?

http://www1.minpic.de/bild_anzeigen.php?id=114417&key=82156785&ende

Greeting Andreas.

Arne Vajhøj · Jun 11, 2010

I hope the pictures are clear.
http://www1.minpic.de/bild_anzeigen.php?id=114414&key=77476860&ende

http://www1.minpic.de/bild_anzeigen.php?id=114415&key=92154987&ende

First, I measure my 3 or more points.
Second, I check the units(steps) to the axis. (Step motor)
Third, I should calculate the coefficient.

How is the best, easiest way?

Thanks for tipps.

--Simple linear regression, or?

Yes.

Simple linear regression.

http://www.mathdotnet.com/

can do it.

Some old code from the shelf:

private static void Solve(double[] y, out double[] b, double[] x)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Matrix x3 = Matrix.CreateFromColumns(new List<Vector> {
one, x2 });
Vector y2 = Vector.Create(y);
Matrix y3 = Matrix.CreateFromColumns(new List<Vector> { y2 });
Matrix b3 = x3.Solve(y3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}

Arne

Andreas Bauer · Jun 11, 2010

Hello Arne,

--Simple linear regression, or?

Click to expand...

Yes.

Simple linear regression.

http://www.mathdotnet.com/

can do it.

Some old code from the shelf:

private static void Solve(double[] y, out double[] b, double[] x)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Matrix x3 = Matrix.CreateFromColumns(new List<Vector> { one,
x2 });
Vector y2 = Vector.Create(y);
Matrix y3 = Matrix.CreateFromColumns(new List<Vector> { y2 });
Matrix b3 = x3.Solve(y3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}

it is not clear.
On my Excel sheet, you see may problem

Two axis

x
y

Customer insert the values in mm, axis controller needs steps.
First I should calibrate the system.
http://www.fileuploadx.de/49586

How?
How I get a,b and the offset c?

Thanks.
Regards Andreas

Arne Vajhøj · Jun 12, 2010

Hello Arne,

--Simple linear regression, or?

Click to expand...

Yes.

Simple linear regression.

http://www.mathdotnet.com/

can do it.

Some old code from the shelf:

private static void Solve(double[] y, out double[] b, double[] x)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Matrix x3 = Matrix.CreateFromColumns(new List<Vector> { one, x2 });
Vector y2 = Vector.Create(y);
Matrix y3 = Matrix.CreateFromColumns(new List<Vector> { y2 });
Matrix b3 = x3.Solve(y3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}

Click to expand...

it is not clear.
On my Excel sheet, you see may problem

Two axis

x
y

Customer insert the values in mm, axis controller needs steps.
First I should calibrate the system.
http://www.fileuploadx.de/49586

How?
How I get a,b and the offset c?

If you have 3 values for each of fx and fy, then it is
simple linear regression.

Arne

Andreas Bauer · Jun 12, 2010

Hello Arne,

If you have 3 values for each of fx and fy, then it is
simple linear regression.

sorry.
http://www.fileuploadx.de/492401

First I have a axis x
The user want to insert the values in mm.
But I should send in C++ or in C# the steps,units to the x controller
(Step motor)

In C# I found a code, but I'm not sure is working or not.
[Linear regression of polynomial coefficients]
http://www.trentfguidry.net/post/2009/08/01/Linear-regression-polynomial-coefficients.aspx

[Linear and multiple linear regression]
http://www.trentfguidry.net/post/2009/07/19/Linear-multiple-regression.aspx

I should also to understand the algorithm by hand. That is my problem.
Where is a good instruction.
Maybe you can help me.

Thanks a lot.

Regards Andreas

For a first order polynomial (a line), the equation is:
Y = A + BX
For this equation, z0 is 1 and z1 is X, and y is Y.
This is done in the code shown below.
double[] x = new double[] { 2.3601, 2.3942, 2.4098, 2.4268,
2.4443, 2.4552, 2.4689, 2.4885, 2.5093, 2.5287 };
double[] y = new double[] { 133.322, 666.612, 1333.22,
2666.45, 5332.9, 7999.35, 13332.2, 26664.5, 53329, 101325 };
int nPolyOrder = 1;
double[,] z = new double[y.Length, nPolyOrder + 1];
for (int i = 0; i < y.Length; i++)
{
z[i, 0] = 1.0;
z[i, 1] = x;
}
double[] coefs = Polynomial.Regress(z, y);

public static double[] Regress(double[,] z, double[] y)
{
//y=a0 z1 + a1 z1 +a2 z2 + a3 z3 +...
//Z is the functional values.
//Z index 0 is a row, the variables go across index 1.
//Y is the summed value.
//returns the coefficients.
Debug.Assert(z != null && y != null);
Debug.Assert(z.GetLength(0) == y.GetLength(0));

Matrix zMatrix = z;
Matrix zTransposeMatrix = zMatrix.Transpose(); KENNZEICHENooooo
Matrix leftHandSide = zTransposeMatrix * zMatrix;
Matrix rightHandSide = zTransposeMatrix * y;
Matrix coefsMatrix = leftHandSide.SolveFor(rightHandSide);
KENNZEICHENooooo

return coefsMatrix;
}

Andreas Bauer · Jun 12, 2010

http://www1.minpic.de/bild_anzeigen.php?id=114609&key=55273016&ende

Hello,
the next problem is, I dimension x and y.
The new x, ist depend from y.

Measure
mm x mm y Units X Units Y
173,000 12,500 130,816 266,000
275,500 28,500 196,860 256,000
139,500 153,000 113,987 175,000

Greeting Andreas

Arne Vajhøj · Jun 13, 2010

sorry.
http://www.fileuploadx.de/492401

Ah. Units x is fx.

Then it is a very simple linear regression.

using System;
using System.Collections.Generic;

using MathNet.Numerics.LinearAlgebra;

namespace E
{
public class Program
{
private static void Solve(double[] f, out double[] b, double[]
x, double[] y)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Vector y2 = Vector.Create(y);
Matrix xy3 = Matrix.CreateFromColumns(new List<Vector> {
x2, y2, one });
Vector f2 = Vector.Create(f);
Matrix f3 = Matrix.CreateFromColumns(new List<Vector> { f2 });
Matrix b3 = xy3.Solve(f3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}
public static void Main(string[] args)
{
double[] fx = { 130.816, 196.860, 113.987 };
double[] fy = { 266.000, 256.000, 175.000 };
double[] x = { 173.0, 275.5, 139.5 };
double[] y = { 12.5, 28.5, 153-0 };
double[] b;
Solve(fx, out b, x, y);
Console.WriteLine("fx = " + b[0] + "*x + " + b[1] + "*y + "
+ b[2]);
Solve(fy, out b, x, y);
Console.WriteLine("fy = " + b[0] + "*x + " + b[1] + "*y + "
+ b[2]);
Console.ReadKey();
}
}
}

outputs:

fx = 0,639237208990945*x + 0,0326366299017556*y + 19,8200049707945
fy = 0,00341428308423573*x + -0,646872751008385*y + 273,495238414032

which looks as the results you have found in Excel.

Arne

Andreas Bauer · Jun 13, 2010

Hello Arne,

Then it is a very simple linear regression.

using System;
using System.Collections.Generic;

using MathNet.Numerics.LinearAlgebra;

namespace E
{
public class Program
{
private static void Solve(double[] f, out double[] b, double[]
x, double[] y)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Vector y2 = Vector.Create(y);
Matrix xy3 = Matrix.CreateFromColumns(new List<Vector> { x2,
y2, one });
Vector f2 = Vector.Create(f);
Matrix f3 = Matrix.CreateFromColumns(new List<Vector> { f2 });
Matrix b3 = xy3.Solve(f3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}
public static void Main(string[] args)
{
double[] fx = { 130.816, 196.860, 113.987 };
double[] fy = { 266.000, 256.000, 175.000 };
double[] x = { 173.0, 275.5, 139.5 };
double[] y = { 12.5, 28.5, 153-0 };
double[] b;
Solve(fx, out b, x, y);
Console.WriteLine("fx = " + b[0] + "*x + " + b[1] + "*y + "
+ b[2]);
Solve(fy, out b, x, y);
Console.WriteLine("fy = " + b[0] + "*x + " + b[1] + "*y + "
+ b[2]);
Console.ReadKey();
}
}
}

outputs:

fx = 0,639237208990945*x + 0,0326366299017556*y + 19,8200049707945
fy = 0,00341428308423573*x + -0,646872751008385*y + 273,495238414032

which looks as the results you have found in Excel.

fine, thank you very much.

Two additional question. I hope is ok.
What is exactly the theoretical math. background.
Linear regression? I have a relation from x to y.
Can you show me, if I must calculate by hand, I need the correct way,
the correct math. formular.

My problem.
I have a new task.
I should find a solution.
If I know not the correct headline, is not possible to search via
www.google.com or via www.bing.com

Greeting Andreas

Fred Mellender · Jun 13, 2010

See http://www.easycalculation.com/statistics/learn-regression.php

For each point (x,y) you need to form the sums and products indicated. Then
you can get the equation of the line.

Andreas Bauer said:
Hello Arne,

Then it is a very simple linear regression.

using System;
using System.Collections.Generic;

using MathNet.Numerics.LinearAlgebra;

namespace E
{
public class Program
{
private static void Solve(double[] f, out double[] b, double[] x,
double[] y)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Vector y2 = Vector.Create(y);
Matrix xy3 = Matrix.CreateFromColumns(new List<Vector> { x2,
y2, one });
Vector f2 = Vector.Create(f);
Matrix f3 = Matrix.CreateFromColumns(new List<Vector> {
f2 });
Matrix b3 = xy3.Solve(f3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}
public static void Main(string[] args)
{
double[] fx = { 130.816, 196.860, 113.987 };
double[] fy = { 266.000, 256.000, 175.000 };
double[] x = { 173.0, 275.5, 139.5 };
double[] y = { 12.5, 28.5, 153-0 };
double[] b;
Solve(fx, out b, x, y);
Console.WriteLine("fx = " + b[0] + "*x + " + b[1] + "*y + " +
b[2]);
Solve(fy, out b, x, y);
Console.WriteLine("fy = " + b[0] + "*x + " + b[1] + "*y + " +
b[2]);
Console.ReadKey();
}
}
}

outputs:

fx = 0,639237208990945*x + 0,0326366299017556*y + 19,8200049707945
fy = 0,00341428308423573*x + -0,646872751008385*y + 273,495238414032

which looks as the results you have found in Excel.

Click to expand...

fine, thank you very much.

Two additional question. I hope is ok.
What is exactly the theoretical math. background.
Linear regression? I have a relation from x to y.
Can you show me, if I must calculate by hand, I need the correct way, the
correct math. formular.

My problem.
I have a new task.
I should find a solution.
If I know not the correct headline, is not possible to search via
www.google.com or via www.bing.com

Greeting Andreas

Arne Vajhøj · Jun 14, 2010

Then it is a very simple linear regression.

using System;
using System.Collections.Generic;

using MathNet.Numerics.LinearAlgebra;

namespace E
{
public class Program
{
private static void Solve(double[] f, out double[] b, double[] x,
double[] y)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Vector y2 = Vector.Create(y);
Matrix xy3 = Matrix.CreateFromColumns(new List<Vector> { x2, y2, one });
Vector f2 = Vector.Create(f);
Matrix f3 = Matrix.CreateFromColumns(new List<Vector> { f2 });
Matrix b3 = xy3.Solve(f3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}
public static void Main(string[] args)
{
double[] fx = { 130.816, 196.860, 113.987 };
double[] fy = { 266.000, 256.000, 175.000 };
double[] x = { 173.0, 275.5, 139.5 };
double[] y = { 12.5, 28.5, 153-0 };
double[] b;
Solve(fx, out b, x, y);
Console.WriteLine("fx = " + b[0] + "*x + " + b[1] + "*y + " + b[2]);
Solve(fy, out b, x, y);
Console.WriteLine("fy = " + b[0] + "*x + " + b[1] + "*y + " + b[2]);
Console.ReadKey();
}
}
}

outputs:

fx = 0,639237208990945*x + 0,0326366299017556*y + 19,8200049707945
fy = 0,00341428308423573*x + -0,646872751008385*y + 273,495238414032

which looks as the results you have found in Excel.

Click to expand...

fine, thank you very much.

Two additional question. I hope is ok.
What is exactly the theoretical math. background.
Linear regression? I have a relation from x to y.
Can you show me, if I must calculate by hand, I need the correct way,
the correct math. formular.

The basic formula for linear regression is:

b = inv(x'x)x'y

Read more here:
http://en.wikipedia.org/wiki/Linear_regression

Note that usually the linear regression is not done
with a straight forward inversion but uses a decomposition.

Arne

Andreas Bauer · Jun 14, 2010

Hello Arne,
works well. Thanks a lot. The math. assembly looks well.
The link it is not easy to understand all.
b = inv(x'x)x'y

I hope the last questions.
Why this way? This way is correct, I'm sure.
StepsToAxis<=> UserCoordinatesInMM
fx = a*x + b*y + c
and not the simplest way
fx = a*x + b

How is call in mathematics?
Both 'Linear Regression' ?

How looks the calculation, if y not depend from x.
Can you make an example? Maybe, I hope then is clear for me, to see the
differents.

How is the best way to Round? Which function?
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX);
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX + 0.5);

I can calculate this way, it is easy
stepsToAxisX = b[0] * userX + b[1] * userY + b[2];
If I want to use the matrix
How is here the best way? Can you make a simple example?
For example, I have 10 new values, I know the factors b[0] b[1] b[2];
10
20
30
40
50
....
100

User in MM ---- Black Box ----- Steps to axis, axis move

Picture, calculate by hand.
http://www1.minpic.de/bild_anzeigen.php?id=114769&key=72372343&ende

Excel sheet
See sheet - Three Variables are not know
http://www.fileuploadx.de/665288

Greeting Andreas

http://www.mathdotnet.com/downloads/Iridium-2008-8-16-470.ashx

fx = 0,639237208990945*x + 0,0326366299017556*y + 19,8200049707945
fy = 0,00341428308423573*x + -0,646872751008385*y + 273,495238414032

fx = a*x + b*y + c

using MathNet.Numerics.LinearAlgebra; // Assembly notwendig.
private void btnCalc_Click(object sender, EventArgs e)
{
double[] fx = { 130.816, 196.860, 113.987 };
double[] fy = { 266.000, 256.000, 175.000 };
double[] x = { 173.0, 275.5, 139.5 };
double[] y = { 12.5, 28.5, 153};

double[] b;
Solve(fx, out b, x, y);
Console.WriteLine("fx = " + b[0] + "*x + " + b[1] + "*y + "
+ b[2]);
// Steps to axis x
double userX = 40.4;
double userY = 90.5;
double stepsToAxisX = b[0] * userX + b[1] * userY + b[2];
userX = 173.0;
userY = 12.5;
stepsToAxisX = b[0] * userX + b[1] * userY + b[2];
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX);

Solve(fy, out b, x, y);
Console.WriteLine("fy = " + b[0] + "*x + " + b[1] + "*y + "
+ b[2]);
//Console.ReadKey();

}

private static void Solve(double[] f, out double[] b, double[]
x, double[] y)
{
Vector one = Vector.Ones(x.Length);
Vector x2 = Vector.Create(x);
Vector y2 = Vector.Create(y);
Matrix xy3 = Matrix.CreateFromColumns(new List<Vector> {
x2, y2, one });
Vector f2 = Vector.Create(f);
Matrix f3 = Matrix.CreateFromColumns(new List<Vector> { f2 });
Matrix b3 = xy3.Solve(f3);
Vector b2 = b3.GetColumnVector(0);
b = b2.CopyToArray();
}

Arne Vajhøj · Jun 15, 2010

Hello Arne,
works well. Thanks a lot. The math. assembly looks well.
The link it is not easy to understand all.
b = inv(x'x)x'y

fx = a*x + b*y + c

You need to read what b, x and y are in that formula.

formula y = your fx
formula x = [your x your y all 1's]
formula b = [your a your b your c]

Arne

Andreas Bauer · Jun 15, 2010

Hello Arne,

formula y = your fx
formula x = [your x your y all 1's]
formula b = [your a your b your c]

b is clear.

Can you give me an answer for that?

I hope the last questions.
Why this way? This way is correct, I'm sure.
StepsToAxis<=> UserCoordinatesInMM
fx = a*x + b*y + c
and not the simplest way
fx = a*x + b

How is call in mathematics?
Both 'Linear Regression' ?

How is the best way to Round? Which function?
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX);
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX + 0.5);

How is the calculation for more values?

3 Values
Matrix (3,3) * M(3,1) = M(3,1)
- - -
10 Values
Matrix (10,3) * M(3,1) = M(10,1)
or?
n Values
Matrix (n,3) * M(3,1) = M(n,1)

For b vector I need 3 (a,b,c)
How I get it, when I have more values.

Greeting Andreas

Arne Vajhøj · Jun 16, 2010

formula y = your fx
formula x = [your x your y all 1's]
formula b = [your a your b your c]

Click to expand...

b is clear.

Can you give me an answer for that?

I hope the last questions.
Why this way? This way is correct, I'm sure.
StepsToAxis<=> UserCoordinatesInMM
fx = a*x + b*y + c
and not the simplest way
fx = a*x + b

How is call in mathematics?
Both 'Linear Regression' ?

How is the best way to Round? Which function?
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX);
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX + 0.5);

How is the calculation for more values?

3 Values
Matrix (3,3) * M(3,1) = M(3,1)
- - -
10 Values
Matrix (10,3) * M(3,1) = M(10,1)
or?
n Values
Matrix (n,3) * M(3,1) = M(n,1)

For b vector I need 3 (a,b,c)
How I get it, when I have more values.

I have no idea what you are asking about.

Arne

Andreas Bauer · Jun 16, 2010

Hello Arne,

I have no idea what you are asking about.

Arne

Hope now is clear.

Excel Table
http://www.fileuploadx.de/151465

Picture:
http://www1.minpic.de/bild_anzeigen.php?id=114935&key=19057763&ende

first question.
How is the calculation, if I have more then 3 values.

second question.
fx = a*x + b*y + c
this formular should be use, if I have a scale factor and a rotation

third question
fx = a*x + b
this formular should be use, if I have only a scale factor

Both are 'Linear Regression', in mathemaics calls 'Linear
Regression', yes?

Question 4:
How is the best way to Round? Which function do you use?
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX);
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX + 0.5);
I need a integer to my axis.

With best regards Andreas

Andreas Bauer · Jun 19, 2010

Hello Arne,

any tip for me.

Greeting Andreas

Hope now is clear.

Excel Table
http://www.fileuploadx.de/151465

Picture:
http://www1.minpic.de/bild_anzeigen.php?id=114935&key=19057763&ende

first question.
How is the calculation, if I have more then 3 values.

second question.
fx = a*x + b*y + c
this formular should be use, if I have a scale factor and a rotation

third question
fx = a*x + b
this formular should be use, if I have only a scale factor

Both are 'Linear Regression', in mathematics calls 'Linear
Regression', yes?

Question 4:
How is the best way to Round? Which function do you use?
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX);
int stepsToAxisXInteger = Convert.ToInt32(stepsToAxisX + 0.5);
I need a integer to my axis.

With best regards Andreas

Regression - Instruction

Andreas Bauer

Arne Vajhøj

Andreas Bauer

Arne Vajhøj

Andreas Bauer

Arne Vajhøj

Andreas Bauer

Andreas Bauer

Arne Vajhøj

Andreas Bauer

Fred Mellender

Arne Vajhøj

Andreas Bauer

Arne Vajhøj

Andreas Bauer

Arne Vajhøj

Andreas Bauer

Andreas Bauer