EzDevInfo.com

how to minimize a function using Deap?

I need to minimize a function using genetic algorithm and PSO.

Different posts suggest to use DEAP (I am using python) but I do not even understand how to start.

We can consider for example f on the interval i

i=arange(-10,10,0.1)
def f(x):
    return x*sin(x)

How can I minimize this function using DEAP?

Fitness sharing in DEAP

Is there any way to implement fitness sharing/niching using DEAP? Specifically I'm looking for an implementation of the method defined here (Goldberg's fitness sharing) on page 98 of the pdf. If you know of any other methods that are in DEAP, that would be useful as well.

Thanks

Using DEAP for a tetris AI - no convergence?

I am trying to make a tetris ai with a genetic algorithm. Basically it runs a few games for each set of weights and gets a score based on lines cleared and blocks placed. I rolled my own basic mating, mutating, and crossing over function, but it is very slow and not very effective, so I began using the DEAP module.

The problem is is that there is no "perfect" set of weights like in all the examples using deap I have seen. Is there a way to use deap that just progressively improves between multiple runs without excepting convergence to a set of weights?

python DEAP genetic algorithm multi-core speed

I am using Python's DEAP pacakge and I want to multi-core my code and I used the tutorial at http://deap.gel.ulaval.ca/doc/dev/tutorials/distribution.html to successfully do it using multiprocessing.

My question is the following: using 8 cores, how much speedup to I get in theory? The reason I ask is because I want to decide how many individuals and generations I can run in the same amount of time as the single-cored version. My code used to take ~200s to run and with 8 cores, it now takes ~0.5 seconds (this is a 400X speedup). Can I assume that anything will be sped up by 400X? I know it's complex, but your help will be very appreciated.

In general, if anybody can help, I wanted to understand how multicoring changes the flow of computation. Does it just map the evaluation of each individual over different cores for each generation? Or does it run generations in parallel? If you know of any documentation I could read about this, please let me know.

I did not provide a code example as it seems not necessary because this is a very high-level question.

How to fix "gp.generate tried to add a primitive but there is none available"?

I'm trying to construct a typed genetic programming solution with DEAP.

I start the program with a photo of a black & white triangle and the 3 vertices of a different triangle. The hope is for the program to come up with a program which moves the vertices of the given triangle closer to the one in the photo. I provide it random constants, arithmetic primitives add, subtract, etc., if-then-else. and tests for is_black and is_white at given coordinates.

I've set up all my primitives but I keep running into this error, which seems to be telling me to add more primitives which either provide or consume (not sure which?!) the photo.

I find the error difficult because:

• I have primitives which use the photo as input.
• I do not have primitives which produce the photo as output.
• I don't want to morph the photo, just consume it.

I'm thinking the winning program will be a long sequence of commands like "if (10,10) is black add (3,2) to vertex 1" repeated somewhat nauseatingly.

But there is perhaps something I'm not understanding correctly about how deap works. How do I overcome error messages like this?

  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/random.py", line 255, in choice
    raise IndexError('Cannot choose from an empty sequence')
IndexError: The gp.generate function tried to add a primitive of type '<class 'triangles.Photo'>', but there is none available.

This error typically fires from gp.gengrow when it is trying to generate the starting population.

What may/may not be related is that I don't have much in the way of Terminals: I'm not clear how they apply to my problem.

I expect to hear crickets, but if anyone actually takes an interest in this question and wants to see code, or at least the primitive set, I can tack it on or stick it somewhere. Figured it was rambling enough already; whilst I've focused on a specific error message I expect it is moreso my general (un)appreciation of the workings of GP/DEAP that is at fault.

Python - Multiprocessing and shared memory [duplicate]

I am implementing a genetic algorithm using Deap framework. The algorithm works, but I noticed that the multi-process version of the GA is very memory consuming 9 GB, against the 2 GB of the single-process and I suspect because it has been allocate memory for each process. In fact as soon as the map is executed the memory used increases. Since data shared among processes are used only to be read, all of them can access to the same memory.

This is the structure of my code.

def evaluate(individual, dataset=None):

    penalty = dataset.compute(individual)

    return penalty


def initialize():
   dataset = dataset(file1, file2)

   pool = multiprocessing.Pool()
   toolbox.register("map", pool.map)

   toolbox.register("evaluate", evaluate, dataset=dataset)

   return toolbox, dataset


def main():
   toolbox, dataset = initialize()

   dataset.data = some_training_set

   fitnesses = toolbox.map(toolbox.evaluate, population)

   dataset.data = some_validation_set

   fitnesses = toolbox.map(toolbox.evaluate, population)

Then I have a class containing the dataset (read by using pandas) and a dictionary.

class Dataset:

    def __init__(self, file1, file2):
        self.data = read(file1)
        self.dict = loadpickle(file2)

    def compute(self, individual):
       for row in self.data
           # some stuff reading row and self.dict

What is the easiest way to share the memory? I tried to use global variables for self.data and self.dict, but nothing...

Writing my first genetic algorithm using DEAP - Stuck with a Type error

I just started with my first GA algorithm using DEAP. As a beginner I am just trying to DEAPify an algorithm which I have already coded using python. When I try to evaluate my fitness functions by registering in the toolbox container I get the following errors

![Error List][1]

    Traceback (most recent call last):
    fitnesses = toolbox.map(toolbox.evaluate, invalid_ind)
    File "number_optimize_DEAP.py", line 38, in grade
        summed = reduce(add, (fitness(x) for x in population),0)
    File "number_optimize_DEAP.py", line 38, in <genexpr>
        summed = reduce(add, (fitness(x) for x in population),0)
    File "number_optimize_DEAP.py", line 29, in fitness
        sum = reduce(add,individual,0)
    TypeError: reduce() arg 2 must support iteration'

Please have a look at my functions

def fitness (individual):

    sum = reduce(add,individual,0)

    return abs(target - sum)

'Calculating the average population fitness'

def grade(population):

    summed = reduce(add, (fitness(x) for x in population),0)
    return summed / (len(population) * 1.0)

toolbox.register("evaluate",grade)
toolbox.register("Crossover",tools.cxOnePoint)
toolbox.register("Mutate",tools.mutUniformInt,indpb = 0.01)
toolbox.register("Selection",tools.selBest)

def evolution():
    pop = toolbox.Population(n = 100)
    hof = tools.HallOfFame(1)
    stats = tools.Statistics(lambda ind: ind.fitness.values)
    stats.register("avg", numpy.mean)
    stats.register("min",numpy.min)
    stats.register("max", numpy.max)

pop,logbook = algorithms.eaSimple(pop,toolbox, cxpb = 0.2, mutpb = 0.01, ngen = 10, stats         = stats, halloffame = hof, verbose = True)

'Fitness functions calculates the sum of the elements in the individual and looks at how far its away from the target(globally declared)'

'Grade calculates the average fitness'

Please help me out from this !! thanks in advance

Python: DEAP: programatically handle number of func parameters

In the evaluate function (for a genetic programming symbolic regression problem with binary input/output), I want to be able to programmatically handle functions that have different numbers of parameters. (I have the rest of the code set up so that everything auto-adjusts depending on how many columns are in the sample data. The input is a numpy.ndarray (matrix)). How would I do this with the code below without making several if statements? (In the code below, func takes 3 inputs and the 4th column is the answer but any other run may have a different number of inputs. I want to handle this automatically.)

# EVALUATOR
def evalSymbReg(individual):
    # Transform the tree expression in a callable function
    numOfNodes = len(individual)
    func = toolbox.compile(expr=individual)
    # diff = the number wrong / the total number of questions
    diff = (numpy.sum((func(inputDataTransposed[0], inputDataTransposed[1], inputDataTransposed[2]) - inputDataTransposed[3])**2).astype(numpy.float64) / len(inputDataTransposed[0]))
    return diff,

Python: networkx: How to make node size auto-expand to fit the label

I'm using this bit of code from a deap symbolic regression example problem and the graph displays fine but I want the nodes to expand as rounded rectangles to fit the text automatically. (I don't want to just specify the node size through trial and error). How would I do that?

# show tree
import matplotlib.pyplot as plt
import networkx

nodes, edges, labels = gp.graph(bests[0])
graph = networkx.Graph()
graph.add_nodes_from(nodes)
graph.add_edges_from(edges)
pos = networkx.graphviz_layout(graph, prog="dot")

plt.figure(figsize=(7,7))
networkx.draw_networkx_nodes(graph, pos, node_size=900, node_color="w")
networkx.draw_networkx_edges(graph, pos)
networkx.draw_networkx_labels(graph, pos, labels)
plt.axis("off")
plt.show()

Why can't pip install the latest version of deap?

Pypi says there is a new version on their website.

pip search tells me I have an old version and there is a new one.

(Canopy 64bit) Johns-iMac:~ john$ pip search deap
deap                      - Distributed Evolutionary Algorithms in Python
 INSTALLED: 1.0.1
 LATEST:    1.0.2

But pip install -U with all the overrides says there is no new version.

(Canopy 64bit) Johns-iMac:~ john$ pip install -U --allow-unverified --allow-external deap==1.0.2
Collecting deap==1.0.2
Could not find a version that satisfies the requirement deap==1.0.2 (from versions: 0.9.1, 0.9.2, 1.0.0rc3, 1.0.0, 1.0.1)
No distributions matching the version for deap==1.0.2

What silly detail am I overlooking?

Is there an error in the DEAP overview [duplicate]

I'm trying to try out the DEAP package for python and since I'm not too good in python yet I'm taking baby steps. I'm following along with the overview page and I think I've found an error.

On this site http://deap.readthedocs.org/en/1.0.x/overview.html#algorithms

It seems that this line is in error:

for child1, child2 in zip(offspring[::2], offspring[1::2]):

The variable offspring is a map object so I get an object is not subscriptable error. Is the error mine or is this bad code?

Deap: Want to know the generation that created the best individual

I'm running a genetic algorithm program and can find the best individual at the end of the run (hof[0]), but i want to know which generation produced it. Is there any attributes of hof[0] that will help print the individual and the generation that created it. I tried looking at the manuals and Google for answers but could not find it anywhere. I also couldn't find a list of the attributes of individuals that I could print. Can someone point to the right link and documentation to that.

Thanks