<!--

  thr_elements.xml            thelma package documentation          Victor Bovdi
																	Vasyl Laver

  Copyright (C) 2018  Victor Bovdi, Vasyl Laver, Department of Mathematical Sciences,
  UAEU, Al Ain, United Arab Emirates
  This file is free software, see license information at the end.

  This chapter contains the operations with threshold elements
-->

<Chapter Label="thr_elements">
<Heading>Threshold Elements</Heading>

<Section Label="thr_basic">
<Heading>Basic Operations</Heading>

For a given real vector <Alt Only="LaTeX"><M>w=(w_1, \ldots, w_n) \in {\mathbb{R}}^n</M></Alt> and a threshold  <Alt Only="LaTeX"><M>T \in \mathbb{R}</M></Alt>, the <A>threshold element</A> is a function <M>f: \mathbb{Z}_2^n \to \mathbb{Z}_2</M> defined by the following relations:

<Display>
f(x_1,\dots,x_n) = 1, \quad \textrm{if} \quad \sum_{i = 1}^{n} w_i x_i \geq T, \quad \textrm{and} \quad f(x_1,\dots,x_n) = 0 \quad \textrm{otherwise},
</Display>

in which <M>f(x_1,\dots,x_n)</M> is the binary output (valued 0 or 1), each variable <M>x_i</M> is the i-th input (valued 0 or 1), and <M>n</M> is the number of inputs. <P/>

The vector <M>w</M> is the <A>weight</A> vector, and the <M>x=(x_1, \ldots, x_n)</M> is the <A>input</A> vector.
The vector <M>(w_1, \ldots, w_n;T)</M> is called the <A>structure vector</A> (or simply the
<A>structure</A>) of the threshold element.

<ManSection>
<Func Name="ThresholdElement" Arg="Weights, Threshold"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->
For the list of rational numbers <C>Weights</C> and the rational <C>Threshold</C>
the function <C>ThresholdElement</C> returns a threshold element with the
number of inputs equal to the length of the <C>Weights</C> list.

<Example>
<![CDATA[
gap> te:=ThresholdElement([1,2],3);
< threshold element with weight vector [ 1, 2 ] and threshold 3 >
gap> Display(te);
Weight vector = [ 1, 2 ], Threshold = 3.
Threshold Element realizes the function f :
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 0
[ 1, 0 ] || 0
[ 1, 1 ] || 1
Sum of Products:[ 3 ]
]]>
</Example>

The function <C>Display</C> outputs the stucture of the given threshold element <C>ThrEl</C> and
the Sum of Products or Product of Sums representation of the function realized by <C>ThrEl</C>.
For threshold elements of <M>n \leq 4</M> variables it also prints the truth table of the realized Boolean function.

<Example>
<![CDATA[
gap> w:=[1,2,4,-4,6,8,10,-25,6,32];;
gap> T:=60;;
gap> te:=ThresholdElement(w,T);
< threshold element with weight vector [ 1, 2, 4, -4, 6, 8, 10, -25, 6, 32
 ] and threshold 60 >
gap> Display(te);
Weight vector = [ 1, 2, 4, -4, 6, 8, 10, -25, 6, 32 ], Threshold = 60.
Threshold Element realizes the function f :
Sum of Products:[ 59, 155, 185, 187, 251, 315, 379, 411, 427, 441, 443, 507, 5\
71, 667, 697, 699, 763, 827, 891, 923, 939, 953, 955, 1019 ]
]]>
</Example>
</Description> </ManSection>


<ManSection>
<Func Name="IsThresholdElement" Arg="Obj"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

For the object <C>Obj</C> the function <C>IsThresholdElement</C> returns <C>true</C> if
<C>Obj</C> is a threshold element (see <Ref Func="ThresholdElement" />), and <C>false</C> otherwise.

<Example>
<![CDATA[
gap> te:=ThresholdElement([1,2],3);
< threshold element with weight vector [ 1, 2 ] and threshold 3 >
gap> IsThresholdElement(te);
true
gap> IsThresholdElement([[1,2],3]);
false
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="OutputOfThresholdElement" Arg="ThrEl"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->
For the threshold element <C>ThrEl</C> the function <C>OutputOfThresholdElement</C> returns the
Boolean function, realized by <C>ThrEl</C>.

<Example>
<![CDATA[
gap> te:=ThresholdElement([1,2],3);
< threshold element with weight vector [ 1, 2 ] and threshold 3 >
gap> f:=OutputOfThresholdElement(te);
< Boolean function of 2 variables >
gap> Display(f);
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 0
[ 1, 0 ] || 0
[ 1, 1 ] || 1
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="StructureOfThresholdElement" Arg="ThrEl"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

For the threshold element <C>ThrEl</C> the function <C>StructureOfThresholdElement</C> returns the
structure vector [<C>Weights</C>,<C>Threshold</C>] (see <Ref Func="ThresholdElement" />).

<Example>
<![CDATA[
gap> te:=ThresholdElement([1,2],3);
< threshold element with weight vector [ 1, 2 ] and threshold 3 >
gap> sv:=StructureOfThresholdElement(te);
[ [ 1, 2 ], 3 ]
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="RandomThresholdElement" Arg="NumVar, Lo, Hi"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

For the integers <C>NumVar</C>, <C>Lo</C>, and <C>Hi</C>, the function <C>RandomThresholdElement</C> returns
a threshold element of <C>NumVar</C> variables with a pseudo random integer weight vector and an integer threshold,
where both the weights and the threshold are chosen from the interval [<C>Lo</C>, <C>Hi</C>].

<Example>
<![CDATA[
gap> te:=RandomThresholdElement(4,-10,10);
< threshold element with weight vector [ 7, -8, -6, 10 ] and threshold 2 >
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="Comparison of Threshold Elements" Arg="ThrEl1, ThrEl2"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

Let <C>ThrEl1</C> and <C>ThrEl2</C> be two threshold elements of the same number of variables,
which realize the following Boolean functions (see <Ref Func="ThresholdElement"/>) <M>f_1</M> and <M>f_2</M>,
resprectively.
By comparison of two threshold elements we mean the comparison of the truth vectors of <M>f_1</M> and <M>f_2</M>
(see <Ref Func="OutputOfThresholdElement"/>).

<Example>
<![CDATA[
gap> te1:=ThresholdElement([1,2],3);;
gap> Print(OutputOfThresholdElement(te1),"\n");
[2, 2,[ 0, 0, 0, 1 ]]
gap> te2:=ThresholdElement([1,2],0);;
gap> Print(OutputOfThresholdElement(te2),"\n");
[2, 2,[ 1, 1, 1, 1 ]]
gap> te3:=ThresholdElement([1,1],2);;
gap> Print(OutputOfThresholdElement(te3),"\n");
[2, 2,[ 0, 0, 0, 1 ]]
gap> te1<te2;
true
gap> te1>te2;
false
gap> te1=te3;
true
]]>
</Example>
</Description> </ManSection>
</Section>

<Section Label="ste_real">
<Heading>Single Threshold Element Realizability</Heading>

One of the most important questions is whether a Boolean function can be realized by a single threshold element (STE). A Boolean function which is realizable by a STE is called a <C>Threshold Function</C>. This section is dedicated to verification of STE-realizability.<P/>

<ManSection>
<Func Name="CharacteristicVectorOfFunction" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->
Let <M>f(x_1,\ldots,x_n)</M> be a Boolean function. We can switch from the {0,1}-base to {-1,1}-base using the following transformation:
<Display>
y_i = 2x_i-1, \quad (i = 1,2,\ldots,n)
</Display>
<Display>
g(y_1,\ldots,y_n) = 2f(x_1,\ldots,x_n)-1.
</Display>
For each <M>i \in \{1,2,\ldots,n\}</M> the <M>i</M>-th column of the truth table of the function <M>g(y_1,\ldots,y_n)</M>
(in {-1,1}-base) we denote by <M>Y_i</M>, and the truth vector of <M>g</M> we denote by <M>G</M>.<P/>

Define the following vector:
<Display>
b = \big(\;Y_1 \cdot G,\; \ldots, \; Y_n \cdot G, \; \textstyle \sum_{i=0}^{2^n-1} g(i) \; \big) \in \mathbb{R}^{n+1},
</Display>
where <M>Y_k \cdot G</M> is the classical inner (scalar) product for each <M>k \in \{1,\ldots,n\}</M>. <P/>
Vector <M>b</M> is called the <A>characteristic vector</A> of the Boolean function <M>f</M>  <Cite Key="Dertouzos65"/>.
Comparing the characteristic vector of the function <M>f</M> with the lists of characteristic vectors of all STE-realizable functions
we obtain the answer wheter <M>f</M> is realizable by STE or not. In <Package>Thelma</Package> package we have a database of all
such vectors for STE-realizable functions of <M>n \leq 6</M> variables obtained from <Cite Key="Dertouzos65"/>.

For the Boolean function <C>Func</C> the function <C>CharacteristicVectorOfFunction</C> returns a characteristic vector.
There are no limitations on the cardinality of <C>Func</C>, but the database of STE-realizable functions is given only for <M>n \leq 6</M> variables.

<Example>
<![CDATA[
gap> f:=LogicFunction(2,2,[0,0,0,1]);
< Boolean function of 2 variables >
gap> CharacteristicVectorOfFunction(f);
[ 2, 2, 2 ]
]]>
</Example>

</Description> </ManSection>

<ManSection>
<Func Name="IsCharacteristicVectorOfSTE" Arg="ChVect"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

For the characteristic vector <C>ChVect</C> (see <Ref Func="CharacteristicVectorOfFunction"/>) the function <C>IsCharacteristicVectorOfSTE</C>
returns <C>true</C> if <C>ChVect</C> is a characteristic vector of some STE-realizable Boolean function, and <C>false</C> otherwise.
Note, that this function is implemented only for characteristic vectors of length not bigger than 7.

<Example>
<![CDATA[
gap> f:=LogicFunction(2,2,[0,0,0,1]);
< Boolean function of 2 variables >
gap> c:=CharacteristicVectorOfFunction(f);
[ 2, 2, 2 ]
gap> IsCharacteristicVectorOfSTE(c);
true
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="IsInverseInKernel" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

Let <M>f(x_1,\ldots,x_n)</M> be a Boolean function with the kernel <M>K(f)</M>. The function <C>IsInverseInKernel</C> returns <C>true</C> if there is a pair of additive inverse vectors in <M>K(f)</M> (this means that <M>f</M> is not STE-realizable, see <Cite Key="GecheRobotyshyn83"/>) or <C>false</C> otherwise.
Note that this function also accepts the kernel of the Boolean function <C>Func</C> as an input.


A vector <M>b \in \mathbb{Z}_2^n</M> is called an additive inverse to <M>a \in \mathbb{Z}_2^n</M> if <M>a \oplus b = 0</M>. <P/>

<Example>
<![CDATA[
gap> f:=LogicFunction(3,2,[0,1,0,1,0,1,1,0]);;
gap> k:=KernelOfBooleanFunction(f);;
gap> Display(k[1]);
 . . 1
 . 1 1
 1 . 1
 1 1 .
gap> IsInverseInKernel(f);
true
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="IsKernelContainingPrecedingVectors" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

A vector <M>a=(\alpha_1,\ldots,\alpha_n) \in \mathbb{Z}_2^n</M> precedes a vector <M>b=(\beta_1,\ldots,\beta_n) \in \mathbb{Z}_2^n</M>  (we denote it as <M>a \prec b</M>) if <M>\alpha_i \leq \beta_i \textrm{ for each } i=1,\ldots, n</M>. <P/>
For a given vector <M>c \in \mathbb{Z}_2^n</M> denote <M>M_c=\{\;a\in \mathbb{Z}_2^n \;\mid\; a \prec c \;\}</M>. <P/>
Let <M>f(x_1,\ldots,x_n)</M> be a Boolean function with reduced kernel <M>T(f)=\{K(f)_j \mid j=1,2,\ldots, m \}</M>. If <M>f</M> is implemented by a single threshold element (STE), then there exists  <M>j \in \{1,\ldots, m \}</M> such that
<Display>
\forall a \in K(f)_j \qquad \textrm{holds} \qquad M_a \subseteq K(f)_j.
</Display>

The function <C>IsKernelContainingPrecedingVectors</C> returns <C>false</C> for a given function <C>Func</C> if <M>Func</M> is not realizable by a single threshold element (see <Cite Key="GecheMulesa2017"/>).
Note that this function also accepts the kernel of the Boolean function <C>Func</C> as an input.

<Example>
<![CDATA[
gap> ##Continuation of the previous example
gap> IsKernelContainingPrecedingVectors(f);
false
]]>
</Example>
</Description> </ManSection>

<ManSection>

<Func Name="IsRKernelBiggerOfCombSum" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

Let <M>f(x_1,\ldots,x_n)</M> be a Boolean function with reduced kernel <M>T(f)</M>.
Denote
<Display>
  k_i^* = \max\big\{ \; \|a\| = \textstyle \sum_{j=1}^m a_j \; \mid \; a = (a_1, \ldots, a_m) \in T(f) \; \big\}, \quad (i=1,\ldots,n)
</Display>
and
<Display>
  k_A^*=\min\big\{\;k_i^* \; \mid \; i=1, 2, \ldots,n \; \big\}.
</Display>

If <M>f</M> is implemented by a single threshold element (STE), then the following condition holds:
<Display>
|A| \geq \sum_{i=0}^{k_A^*} {{k_A^*}\choose{i}},
</Display>
where <M>{{k_A^*}\choose{i}}</M> is the classical binomial coefficient and <M>|A|</M> is the cardinality of <M>A</M>. <P/>

For a given Boolean function <C>Func</C> the function <C>IsRKernelBiggerOfCombSum</C> returns <C>false</C>
if this function is not STE-realizable (see <Cite Key="GecheMulesa2017"/>). Note that this function also accepts
the reduced kernel of the Boolean function <C>Func</C> as an input.


<Example>
<![CDATA[
gap> f:=LogicFunction(2,2,[0,1,1,0]);
< Boolean function of 2 variables >
gap> IsRKernelBiggerOfCombSum(f);
false
]]>
</Example>
</Description> </ManSection>

<ManSection>

<Func Name="BooleanFunctionBySTE" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

For a given Boolean function <C>Func</C> the function <C>BooleanFunctionBySTE</C> determines whether <C>Func</C> is realizable by a single threshold element (STE).
The function returns a threshold element with integer weights and integer threshold.
If <C>Func</C> is not realizable by STE, it returns an empty list [].
The realization of the function <C>BooleanFunctionBySTE</C> is based on algorithms, proposed in <Cite Key="Geche2010"/>.

<Example>
<![CDATA[
gap> f:=LogicFunction(3,2,[1,1,0,0,1,0,0,0]);
< Boolean function of 3 variables >
gap> te:=BooleanFunctionBySTE(f);
< threshold element with weight vector [ -1, -4, -2 ] and threshold -2 >
gap> Display(te);
Weight vector = [ -1, -4, -2 ], Threshold = -2.
Threshold Element realizes the function f :
Boolean function of 3 variables.
[ 0, 0, 0 ] || 1
[ 0, 0, 1 ] || 1
[ 0, 1, 0 ] || 0
[ 0, 1, 1 ] || 0
[ 1, 0, 0 ] || 1
[ 1, 0, 1 ] || 0
[ 1, 1, 0 ] || 0
[ 1, 1, 1 ] || 0
Sum of Products:[ 0, 1, 4 ]
gap> f:=LogicFunction(2,2,[0,1,1,0]);
< Boolean function of 2 variables >
gap> te:=BooleanFunctionBySTE(f);
[  ]
]]>
</Example>
</Description> </ManSection>

<ManSection>

<Func Name="PDBooleanFunctionBySTE" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

Let <M>f(x_1,\ldots,x_n)</M> be a partially defined Boolean function.
We denote by <C>x</C> the positions in truth vector, where <M>f</M> is undefined.
Then <M>f^{-1}</M><C>(x)</C> is the set of Boolean vectors of <M>n</M> variables on which the function is undefined.
The sets <M>f^{-1}(0)</M> and <M>f^{-1}(1)</M> are defined in <Ref Func="KernelOfBooleanFunction" />.
The function <M>f</M> is called a <A>threshold function</A> if there is an <M>n</M>-dimensional real vector
<M>w=(w_1,\ldots,w_n)</M> and a real threshold <M>T</M> such that
<Display>
a \in f^{-1}(1) \quad \Longrightarrow \quad a\cdot w^T \geq T,
</Display>
<Display>
a \in f^{-1}(0)\quad \Longrightarrow \quad a\cdot w^T &lt; T,
</Display>
where <M>a\cdot w^T</M> is the classical inner (scalar) product. <P/>
For the partially defined Boolean function <C>Func</C> (presented as a string, where <C>x</C> presents the undefined values)
the function <C>PDBooleanFunctionBySTE</C> returns a threshold element if <C>Func</C> can be realized by STE and
empty list otherwise.
The realization of the function <C>PDBooleanFunctionBySTE</C> is based on the algorithm, proposed in <Cite Key="GecheRobotyshyn83"/>.
<Example>
<![CDATA[
gap> f:="1x001x0x";
"1x001x0x"
gap> te:=PDBooleanFunctionBySTE(f);
< threshold element with weight vector [ -1, -2, -3 ] and threshold -1 >
gap> Display(te);
Weight vector = [ -1, -2, -3 ], Threshold = -1.
Threshold Element realizes the function f :
Boolean function of 3 variables.
[ 0, 0, 0 ] || 1
[ 0, 0, 1 ] || 0
[ 0, 1, 0 ] || 0
[ 0, 1, 1 ] || 0
[ 1, 0, 0 ] || 1
[ 1, 0, 1 ] || 0
[ 1, 1, 0 ] || 0
[ 1, 1, 1 ] || 0
Sum of Products:[ 0, 4 ]
]]>
</Example>
</Description> </ManSection>


</Section>


<Section Label="thr_iter">
<Heading>Iterative Training Methods</Heading>

<Package>Thelma</Package> also provides a few iterative methods for threshold element training.

<ManSection>
<Func Name="ThresholdElementTraining" Arg="ThrEl, Step, Func, Max_Iter"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

This is a basic iterative method for the perceptron training <Cite Key="Rosenblatt58"/>.
For the threshold element <C>ThrEl</C> (which is an arbitrary threshold
element for the first iteration), the positive integer <C>Step</C> (the value on which we change parameters
while training the threshold element), the Boolean function <C>Func</C> and the positive integer <C>Max_Iter</C> -
the maximal number of iterations, the function <C>ThresholdElementTraining</C> returns a threshold element, realizing <C>Func</C>
(if such threshold element exists).

<Example>
<![CDATA[
gap> f:=LogicFunction(2,2,[0,0,0,1]);
< Boolean function of 2 variables >
gap> te1:=RandomThresholdElement(2,-2,2);
< threshold element with weight vector [ 0, -1 ] and threshold 0 >
gap> Display(OutputOfThresholdElement(te1));
Boolean function of 2 variables.
[ 0, 0 ] || 1
[ 0, 1 ] || 0
[ 1, 0 ] || 1
[ 1, 1 ] || 0
gap> te2:=ThresholdElementTraining(te1,1,f,100);
< threshold element with weight vector [ 2, 1 ] and threshold 3 >
gap> Display(OutputOfThresholdElement(te2));
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 0
[ 1, 0 ] || 0
[ 1, 1 ] || 1
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="ThresholdElementBatchTraining" Arg="ThrEl, Step, Func, Max_Iter"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->
For the threshold element <C>ThrEl</C> (which is an arbitrary threshold
element for the first iteration), the positive integer <C>Step</C> (the value on which we change parameters
while training the threshold element), the Boolean function <C>Func</C>,
and the positive integer <C>Max_Iter</C> - the maximal number of iterations,
the function <C>ThresholdElementTraining</C> returns a threshold element, realizing <C>Func</C>
(if such threshold element exists) via batch training.

<Example>
<![CDATA[
gap> f:=LogicFunction(2,2,[0,0,0,1]);
< Boolean function of 2 variables >
gap> te1:=RandomThresholdElement(2,-2,2);
< threshold element with weight vector [ 0, 2 ] and threshold 2 >
gap> Display(OutputOfThresholdElement(te1));
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 1
[ 1, 0 ] || 0
[ 1, 1 ] || 1
gap> te2:=ThresholdElementBatchTraining(te1,1,f,100);
< threshold element with weight vector [ 2, 2 ] and threshold 3 >
gap> Display(OutputOfThresholdElement(te2));
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 0
[ 1, 0 ] || 0
[ 1, 1 ] || 1
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="WinnowAlgorithm" Arg="Func, Step, Max_Iter"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->
A Boolean function <M>f:\mathbb{Z}_2^n \to \mathbb{Z}_2</M> which can be presented in the following form:
<Display>
f(x_1,\ldots,x_n)=x_{i_1} \vee \cdots \vee x_{i_k}, \qquad (k \leq n)
</Display>
is called a <A>monotone disjunction</A>, i.e. it is a disjunction in which no variable
appears negated. <P/>

If the given Boolean function <M>f</M> is a monotone disjunction, the <A>Winnow algorithm</A>
is more efficient than the classical Perceptron training algorithm <Cite Key="Littlestone88"/>. <P/>

For the Boolean function <C>Func</C>, which is a monotone disjunction, <C>WinnowAlgorithm</C> returns either a threshold element
realizing <C>Func</C> or [] if <C>Func</C> is not trainable by <C>WinnowAlgorithm</C>. The positive ingetger <C>Step</C> which is not equal to 1 defines the value on which we change parameters
while running the algorithm and the positive integer <C>Max_Iter</C> defines the maximal number of iterations.

<Example>
<![CDATA[
gap> x:=Indeterminate(GF(2),"x");;
gap> y:=Indeterminate(GF(2),"y");;
gap> pol:=x*y+x+y;;
gap> f:=PolynomialToBooleanFunction(pol,2);
< Boolean function of 2 variables >
gap> Display(f);
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 1
[ 1, 0 ] || 1
[ 1, 1 ] || 1
gap> te:=WinnowAlgorithm(f,2,100);
< threshold element with weight vector [ 1, 1 ] and threshold 1 >
gap> Display(OutputOfThresholdElement(te));
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 1
[ 1, 0 ] || 1
[ 1, 1 ] || 1
]]>
</Example>
</Description> </ManSection>

<ManSection>
<Func Name="Winnow2Algorithm" Arg="Func, Step, Max_Iter"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->

For any <M>X \subseteq \mathbb{Z}_2^n</M> and for any <M>\delta</M> satisfying
<M>0 &lt; \delta \leq 1</M> let <M>F(X,\delta)</M> be the class of functions from
<M>X</M> to <M>\mathbb{Z}_2^n</M>. Assume that <M>F(X,\delta)</M> satisfies the following condition:<P/>
for each <M>f \in F(X,\delta)</M> there exist <M>\mu_1,\ldots,\mu_n \geq 0</M>
such that for all <M>(x_1,\ldots,x_n) \in X</M>
<Display>
\textstyle  \sum_{i=1}^n \mu_i x_i \geq 1, \quad \textrm{if} \quad f(x_1,\ldots,x_n)=1
</Display>
and
<Display>
\textstyle  \sum_{i=1}^n \mu_i x_i \leq 1, \quad \textrm{if} \quad f(x_1,\ldots,x_n)=0.
</Display>
In other words, the inverse images of 0 and 1 are linearly separable with a minimum separation
that depends on <M>\delta</M>. <C>Winnow2</C> algorithm is designed for training this class of
the Boolean functions <Cite Key="Littlestone88"/>.<P/>

For the Boolean function <C>Func</C> from the class of Boolean functions which is described above,
the function <C>Winnow2Algorithm</C> returns either a threshold element which realizes <C>Func</C> or [] if <C>Func</C> is not trainable by <C>Winnow2Algorithm</C>.
The positive integer <C>Step</C> which is not equal to 1
defines the value on which we change parameters while running the algorithm.


<Example>
<![CDATA[
gap> ##Conjunction can not be trained by Winnow algorithm.
gap> x:=Indeterminate(GF(2),"x");;
gap> y:=Indeterminate(GF(2),"y");;
gap> pol:=x*y;;
gap> f:=PolynomialToBooleanFunction(pol,2);
< Boolean function of 2 variables >
gap> te:=WinnowAlgorithm(f,2,100);
[  ]
gap> ## But in the case of Winnow2 we can obtain the desirable result.
gap> te:=Winnow2Algorithm(f,2,100);
< threshold element with weight vector [ 1/2, 1/2 ] and threshold 1 >
gap> Display(te);
Weight vector = [ 1/2, 1/2 ], Threshold = 1.
Threshold Element realizes the function f :
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 0
[ 1, 0 ] || 0
[ 1, 1 ] || 1
Sum of Products:[ 3 ]
]]>
</Example>
</Description> </ManSection>


<ManSection>
<Func Name="STESynthesis" Arg="Func"/>
<Description>
<!-- The names chosen for the arguments describe their meaning.-->


The function <C>STESynthesis</C> is based on the algorithm proposed in <Cite Key="Dertouzos65"/>.
In each iteration we perturb an <M>n+1</M>-dimensional weight-threshold vector in such manner that
the distance between the given vector and a desired weight-threshold vector, if such vector exists, is reduced.
So if the Boolean function <C>Func</C> is STE-realizable, then this procedure will eventually yield an acceptable
weight-threshold vector. Otherwise iteration process will eventually enter a limit cycle and the execution
of <C>STE_Synthesis</C> will be stopped. <P/>

For the Boolean function <C>Func</C> the function <C>STESynthesis</C> returns a threshold element if
<C>Func</C> is STE-realizable or an empty list otherwise.<P/>


<Example>
<![CDATA[
gap> f:=x*y+x+y;;
gap> x:=Indeterminate(GF(2),"x");;
gap> y:=Indeterminate(GF(2),"y");;
gap> pol:=x*y+x+y;;
gap> f:=PolynomialToBooleanFunction(pol,2);;
gap> te:=STESynthesis(f);
< threshold element with weight vector [ 2, 2 ] and threshold 1 >
gap> Display(te);
Weight vector = [ 2, 2 ], Threshold = 1.
Threshold Element realizes the function f :
Boolean function of 2 variables.
[ 0, 0 ] || 0
[ 0, 1 ] || 1
[ 1, 0 ] || 1
[ 1, 1 ] || 1
Product of Sums:[ 0 ]
]]>
</Example>
</Description> </ManSection>



</Section>
<!-- ############################################################ -->

</Chapter>

<!--
  This program is free software; you can redistribute it and/or modify
  it under the terms of the GNU General Public License as published by
  the Free Software Foundation; version 2 of the License.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with this program; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
-->
