Getting Started in Python

Example starter code*

*Updated 29th April

Please use the following class template as a pattern to encapsulate your submission.

You may modify it extensively, however you must:

• permit the environment object to be passed in the class initialization, and
• implement the "generate" method to return the best policy developed by your algorithm. In this template, you are free to add (but not remove) methods, parameters, etc. and also to set defaults for any of your additional parameters which permit your code to perform at its best. Only the generate method will be called during the evaluation process, and no parameters will be passed.

*Updated 1st July

A proving environment is also available for participants to hone their submission before the 6th July Deadline

Previous environments may be replaced with the class:

ChallengeProveEnvironment()

*Updated Submissions:

Note: your XX.zip file should consist of your single code submission titled submission.py which has within it your CustomAgent() class and a Requirements.txt file which may be auto-generated.

Example submission: submission.py

from sys import exit, exc_info, argv
import numpy as np
import pandas as pd

!pip3 install git+https://github.com/slremy/netsapi --user --upgrade

from netsapi.challenge import *

class CustomAgent:
    def __init__(self, environment):
        self.environment = environment

    def generate(self):
        best_policy = None
        best_reward = -float('Inf')
        candidates = []
        try:
            # Agents should make use of 20 episodes in each training run, if making sequential decisions
            for i in range(20):
                self.environment.reset()
                policy = {}
                for j in range(5): #episode length
                    policy[str(j+1)]=[random.random(),random.random()]
                candidates.append(policy)
                
            rewards = self.environment.evaluatePolicy(candidates)
            best_policy = candidates[np.argmax(rewards)]
            best_reward = rewards[np.argmax(rewards)]
        
        except (KeyboardInterrupt, SystemExit):
            print(exc_info())
            
        return best_policy, best_reward

How to create a local score for your CustomAgent()

*not included in submission.py

EvaluateChallengeSubmission(ChallengeSeqDecEnvironment, CustomAgent, "example.csv") #How scoring files generated by participants should *not* be included in submission.py file

FAQ

Evaluation

Submission method:

Code must be submitted through the competition platform by the deadlines for evaluation. The top 100 participants on the public leaderboard of test submissions will be given access to the final submission phase. Using multiple accounts to increase the number of submissions in NOT permitted. The entries must be formatted as specified on the Competition page. In case of any problems, send email to support@hexagon-ml.com. Competition related questions can be posed at our discussion forum, KDD 2019 Policy Learning For Malaria Elimination.

Awards:

The 10 top ranking final submissions for the KDD Cup|Humanities Track Competition qualify for cash prizes:

1st     $5000

2nd    $4000

3rd     $3000

4th     $3000

5th     $3000

6th     $2000

7th     $2000

8th     $1000

9th     $1000

10th   $1000

There is no other publication requirement. The winners will be required to make their code publicly available under an OSI-approved license such as, for instance, Apache 2.0, MIT or BSD-like license, if they accept their prize, within a week of the deadline for submitting the final results. Entries exceeding the time budget will not qualify for prizes. In case of a tie, the prize will go to the participant who submitted his/her entry first. Non winners or entrants who decline their prize retain all their rights on their entries and are not obliged to publicly release their code.

Rules

Announcements:

To receive announcements and be informed of any change in rules, the participants must provide a valid email to the challenge platform

Conditions of participation:

Participation requires complying with the rules of the Competition. Prize eligibility is restricted by US government export regulations. The organizers, sponsors, their students, close family members (parents, sibling, spouse or children) and household members, as well as any person having had access to the truth values or to any information about the data or the Competition design giving him (or her) an unfair advantage are excluded from participation. A disqualified person may submit one or several entries in the Competition and request to have them evaluated, provided that they notify the organizers of their conflict of interest. If a disqualified person submits an entry, this entry will not be part of the final ranking and does not qualify for prizes. The participants should be aware that the organizers reserve the right to evaluate for scientific purposes any entry made in the challenge, whether or not it qualifies for prizes.

Dissemination:

The Winners will be invited to attend a workshop organized in conjunction with KDD and contribute to the proceedings.

Registration:

The participants must register to the platform and provide a valid email address. Teams must register only once and provide a group email, which is forwarded to all team members. Teams or solo participants registering multiple times to gain an advantage in the competition may be disqualified.

Anonymity:

The participants who do not present their results at the workshop can elect to remain anonymous by using a pseudonym. Their results will be published on the leaderboard under that pseudonym, and their real name will remain confidential. However, the participants must disclose their real identity to the organizers to claim any prize they might win. 

Submission Expectation:

• Your submission is based on 10 instantiations of the environment object
• Each environment has a limit of 105 evaluations, with an episode requiring 5 evaluations
• Your submission should include submission of a learnt policy and it's evaluation, in the provided .csv format
• Best practices and expectations should refer to the example and tutorial code
• Knowledge should not be shared between runs, any violation of this will be removed in the verification phase
• Submissions should be reasonably constrained to standard Python libraries
• If submitted code cannot be run, the team may be contacted, if minor remediation or sufficient information not provided to run the code, the submission will be removed

Leaderboard

Data License

This competition is brought to you by IBM and University of Oxford. 

https://arxiv.org/abs/1712.00428