diff -Nru r-cran-irace-3.4.1/BUGS r-cran-irace-3.5/BUGS --- r-cran-irace-3.4.1/BUGS 2020-03-30 19:46:56.000000000 +0000 +++ r-cran-irace-3.5/BUGS 2022-02-19 09:51:47.000000000 +0000 @@ -1,112 +1,8 @@ ------------------------------------------------------------*- mode: text -*- -#12 ------------------------------------------------------------ - -irace --check or checkIraceScenario do not work with targetRunnerParallel if -the user does not use target.runner at all. - -#11 ------------------------------------------------------------ - -The semantics of recovery are not clear. Should it recover the exact parameters -given to the original run? If so, then moving to a new execdir will mess up -things. Should it allow changing parameters after recovery (that is, what to do -if it finds a new scenario file with different values or it is passed -additional command-line parameters?). - -One possible behavior could be: Recover everything as given originally; if new -settings are given, either by a configuration file or command-line, override -them; for settings that are still set to default values in the original, -recompute the default (because overriden settings may influence the default of -other settings). Report everything that is restored and overriden. - -This means recovery should happen much earlier, before reading -command-line/scenario file. - - -#9 ------------------------------------------------------------ - -By default -# trainInstancesDir = "./Instances/" -# testInstancesDir = "" - -So not specifying any directory, like: - -## File with a list of instances and (optionally) parameters. -## If empty or NULL, do not use a file. -trainInstancesFile = "instances-train.txt" - -## File containing a list of test instances and optionally additional -## parameters for them. If empty or NULL, do not use a file. -testInstancesFile = "instances-test.txt" - -defaults to for train instances to Instances/train-instance-1 ... -but to ./test-instance-1 ... for test instances, which is unexpected. - -# FIXME: Use NA as default value for testInstancesFile/testInstancesDir. -If testInstancesFile != NA and testInstancesDir == NA, -then make testInstancesDir = trainInstancesDir. Maybe give a warning when doing this. - - -#8 ------------------------------------------------------------ - -We do not re-evaluate the elitist (using which.exe), but we do call -target.evaluator on them. See this code in race.R: - - # Execute commands - if (length(which.exe) > 0) { - which.exps <- which(which.alive %in% which.exe) - target.output[which.exps] <- execute.experiments (experiments[which.exps], scenario) - } - irace.assert(!any(sapply(target.output, is.null))) - - # target.evaluator may be NULL. If so, target.output must - # contain the right output already. - if (!is.null(.irace$target.evaluator)) - target.output <- execute.evaluator (experiments, scenario, target.output, - configurations[which.alive, ".ID."]) - -The reason for that is, in some scenarios, we want to normalize the output of -the algorithm, so when we evaluate new configurations, there is a need to -re-normalize the output of the elites as well. So we need to call -target-evaluator again. - -However, when recovering, the files needed by target.evaluator are likely -lost. We would need to call again target-runner. - -This is perhaps a design mistake: We could simply do the normalization internally in irace or create a separate target script (target-re-evaluator?) that only people needing something like re-normalization will use. - -As a short-term fix, we could implement something like: - -If recovering and target-evaluator is in use, keep elites as elites, but re-run target-runner on them. (Perhaps we need a new .irace$ flag saying: first race from recovery). - - -#6 ------------------------------------------------------------ - -Sometimes system2 does not say what exactly failed or what command was -run when something failed, it just prints: "error in running command". -Can we provide more details? - - -#2 ------------------------------------------------------------ - -A parameter like: - -a "" c (0, 5, 10, 20) - -and a condition like a > 10 will return TRUE when a is 5, because a -is actually "5" so the comparison is lexicographic. How to solve this? - - - Fix 1. Check if a can be converted to numeric, then force it before - evaluating conditions. However, we cannot know for sure if the user - actually wants to convert a to numeric or not. Imagine a = "+3", then - a == "+3" would fail! - - - Fix 2. Force users to use quotes when defining categorical and - ordinal parameters. Hopefully this will make obvious that these - values are strings and cannot be compared with other numbers. - -This is already documented in the user-guide. +Please report bugs here: https://github.com/MLopez-Ibanez/irace/issues +This file is only kept for historical purposes. ############################################################### # FIXED bugs Binary files /tmp/tmp8a__4ei2/aE0Hq8Hydw/r-cran-irace-3.4.1/build/partial.rdb and /tmp/tmp8a__4ei2/hWwq4wfXpn/r-cran-irace-3.5/build/partial.rdb differ Binary files /tmp/tmp8a__4ei2/aE0Hq8Hydw/r-cran-irace-3.4.1/build/vignette.rds and /tmp/tmp8a__4ei2/hWwq4wfXpn/r-cran-irace-3.5/build/vignette.rds differ diff -Nru r-cran-irace-3.4.1/cleanup r-cran-irace-3.5/cleanup --- r-cran-irace-3.4.1/cleanup 2020-03-31 08:07:18.000000000 +0000 +++ r-cran-irace-3.5/cleanup 2022-10-23 13:44:18.000000000 +0000 @@ -1,10 +1,10 @@ #!/bin/sh rm -f irace-Ex.R config.* confdefs.h \ - src/*.o src/*.so src/config.h src/symbols.rds \ + src/*.so src/*.o src/*/*.o src/*.gcno src/*/*.gcno src/config.h src/symbols.rds \ inst/doc/*.blg inst/doc/*.bbl \ tests/testthat/*.log tests/testthat/*.Rout tests/testthat/irace.Rdata -find . -name '*.orig' -o -name '.Rhistory' | xargs rm -f +find . -name '*.orig' -o -name '.Rhistory' -o -name '*.rej' | xargs rm -f rm -rf autom4te.cache diff -Nru r-cran-irace-3.4.1/debian/changelog r-cran-irace-3.5/debian/changelog --- r-cran-irace-3.4.1/debian/changelog 2020-05-30 14:14:47.000000000 +0000 +++ r-cran-irace-3.5/debian/changelog 2022-11-01 08:47:28.000000000 +0000 @@ -1,8 +1,17 @@ -r-cran-irace (3.4.1-1build1) groovy; urgency=medium +r-cran-irace (3.5-1) unstable; urgency=medium - * No-change rebuild against r-api-4.0 + [ Debian Janitor ] + * Bump debhelper from old 12 to 13. + * Set upstream metadata fields: Archive, Bug-Database, Bug-Submit, Repository, + Repository-Browse. + * Update standards version to 4.5.1, no changes needed. - -- Graham Inggs Sat, 30 May 2020 14:14:47 +0000 + [ Andreas Tille ] + * New upstream version + * Standards-Version: 4.6.1 (routine-update) + * Reorder sequence of d/control fields by cme (routine-update) + + -- Andreas Tille Tue, 01 Nov 2022 09:47:28 +0100 r-cran-irace (3.4.1-1) unstable; urgency=medium diff -Nru r-cran-irace-3.4.1/debian/control r-cran-irace-3.5/debian/control --- r-cran-irace-3.4.1/debian/control 2020-04-04 07:16:49.000000000 +0000 +++ r-cran-irace-3.5/debian/control 2022-11-01 08:47:28.000000000 +0000 @@ -1,18 +1,18 @@ Source: r-cran-irace -Maintainer: Debian R Packages Maintainers -Uploaders: Andreas Tille Section: gnu-r -Testsuite: autopkgtest-pkg-r Priority: optional -Build-Depends: debhelper-compat (= 12), - dh-r, - r-base-dev, - r-cran-r6 -Standards-Version: 4.5.0 +Maintainer: Debian R Packages Maintainers +Uploaders: Andreas Tille Vcs-Browser: https://salsa.debian.org/r-pkg-team/r-cran-irace Vcs-Git: https://salsa.debian.org/r-pkg-team/r-cran-irace.git Homepage: https://cran.r-project.org/package=irace +Standards-Version: 4.6.1 Rules-Requires-Root: no +Build-Depends: debhelper-compat (= 13), + dh-r, + r-base-dev, + r-cran-r6 +Testsuite: autopkgtest-pkg-r Package: r-cran-irace Architecture: any diff -Nru r-cran-irace-3.4.1/debian/upstream/metadata r-cran-irace-3.5/debian/upstream/metadata --- r-cran-irace-3.4.1/debian/upstream/metadata 1970-01-01 00:00:00.000000000 +0000 +++ r-cran-irace-3.5/debian/upstream/metadata 2022-11-01 08:47:28.000000000 +0000 @@ -0,0 +1,6 @@ +--- +Archive: CRAN +Bug-Database: https://github.com/MLopez-Ibanez/irace/issues +Bug-Submit: https://github.com/MLopez-Ibanez/irace/issues/new +Repository: https://github.com/MLopez-Ibanez/irace.git +Repository-Browse: https://github.com/MLopez-Ibanez/irace diff -Nru r-cran-irace-3.4.1/DESCRIPTION r-cran-irace-3.5/DESCRIPTION --- r-cran-irace-3.4.1/DESCRIPTION 2020-03-31 13:40:03.000000000 +0000 +++ r-cran-irace-3.5/DESCRIPTION 2022-10-23 14:55:06.000000000 +0000 @@ -7,7 +7,7 @@ settings given a set of instances of an optimization problem. M. López-Ibáñez, J. Dubois-Lacoste, L. Pérez Cáceres, T. Stützle, and M. Birattari (2016) . -Version: 3.4.1 +Version: 3.5 Authors@R: c(person("Manuel", "López-Ibáñez", role = c("aut", "cre"), email = "manuel.lopez-ibanez@manchester.ac.uk", comment = c(ORCID = "0000-0001-9974-1295")), @@ -21,19 +21,18 @@ Depends: R (>= 3.2.0) Imports: stats, utils, compiler, R6 Suggests: Rmpi (>= 0.6.0), parallel, knitr, testthat, withr, mlr (>= - 2.15.0), ParamHelpers, devtools + 2.15.0), ParamHelpers, devtools, covr VignetteBuilder: knitr License: GPL (>= 2) -URL: http://iridia.ulb.ac.be/irace, +URL: https://mlopez-ibanez.github.io/irace/, https://github.com/MLopez-Ibanez/irace BugReports: https://github.com/MLopez-Ibanez/irace/issues ByteCompile: yes -LazyData: yes Encoding: UTF-8 -RoxygenNote: 6.1.1 +RoxygenNote: 7.1.1 SystemRequirements: GNU make NeedsCompilation: yes -Packaged: 2020-03-31 08:07:18 UTC; manu +Packaged: 2022-10-23 13:44:18 UTC; manu Author: Manuel López-Ibáñez [aut, cre] (), Jérémie Dubois-Lacoste [aut], @@ -45,4 +44,4 @@ Nguyen Dang [ctb] Maintainer: Manuel López-Ibáñez Repository: CRAN -Date/Publication: 2020-03-31 13:40:03 UTC +Date/Publication: 2022-10-23 14:55:06 UTC diff -Nru r-cran-irace-3.4.1/inst/bin/parallel-irace-mpi r-cran-irace-3.5/inst/bin/parallel-irace-mpi --- r-cran-irace-3.4.1/inst/bin/parallel-irace-mpi 2020-03-30 19:46:56.000000000 +0000 +++ r-cran-irace-3.5/inst/bin/parallel-irace-mpi 2022-05-17 07:48:32.000000000 +0000 @@ -60,14 +60,15 @@ usage() { cat < [--machine MACHINE] [IRACE PARAMS] +Usage: $0 N[-M] [EXECDIR] --parallel NUM [--seed NUM] [--machine MACHINE] [IRACE PARAMS] Parameters: N an integer giving the number of repetitions of irace or a sequence N-M giving which repetitions to redo. EXECDIR job M will use EXECDIR-M directory (default: execdir-) as the execDir (--exec-dir) parameter of irace. - --parallel number of parallel jobs + --parallel NUM number of parallel jobs (value in scenario.txt is ignored) + --seed NUM the seed of each run i will be SEED+i-1 (default: 1234567) --machine MACHINE qsub queue type, e.g., opteron6272 IRACE PARAMS additional parameters for irace. EOF diff -Nru r-cran-irace-3.4.1/inst/bin/parallel-irace-qsub r-cran-irace-3.5/inst/bin/parallel-irace-qsub --- r-cran-irace-3.4.1/inst/bin/parallel-irace-qsub 2020-03-30 19:46:56.000000000 +0000 +++ r-cran-irace-3.5/inst/bin/parallel-irace-qsub 2022-05-17 07:49:21.000000000 +0000 @@ -55,7 +55,7 @@ exit \$RET EOF } -## End of customization +## END OF CUSTOMIZATION error () { echo "$0: error: $@" >&2 @@ -64,13 +64,14 @@ usage() { cat <&2 + exit 1 +} + +usage() { + cat <>= library("irace") load("examples.Rdata") -load("irace-acotsp.Rdata") -load("log-ablation.Rdata") +iraceResults <- irace::read_logfile("irace-acotsp.Rdata") +log_ablation_file <- file.path(system.file(package="irace"), "exdata", "log-ablation.Rdata") +load(log_ablation_file) options(width = 70) @ \newpage @@ -248,7 +253,7 @@ \end{center} % More information about \irace is available at -\url{http://iridia.ulb.ac.be/irace}. +\url{https://mlopez-ibanez.github.io/irace}. \subsection{Version} The current version of the \irace package is \iraceversion. Previous @@ -545,26 +550,13 @@ \end{lstlisting} %@ -\item Copy all the template files from the \IRACEHOME{/templates/} -directory to the scenario directory. -%<>= +\item Initialize the tuning directory with template config files. On GNU/Linux or OS X, you can do this as follows: +%<>= \begin{lstlisting}[style=BashInputStyle] -# $IRACE_HOME is the installation directory of irace. -cp $IRACE_HOME/templates/*.tmpl ~/tuning/ +irace --init \end{lstlisting} %@ -% Remember that \IRACEHOME{} is the path to the installation -% directory of \irace. It can be obtained in the \aR console with: -% -% <>= -% library("irace") -% system.file(package = "irace") -% @ - -\item For each template in your tuning directory, remove the \code{.tmpl} - suffix, and modify them following the next steps. - \item Define the target algorithm parameters to be tuned by following the instructions in \code{parameters.txt}. Available parameter types and other guidelines can be found in \autoref{sec:target parameters}. @@ -613,9 +605,7 @@ time (\code{cost [time]}). When the \parameter{maxTime} option is used, returning \code{time} is mandatory. The \code{target-runner} template is written in \proglang{GNU Bash} scripting language, which can be executed easily in GNU/Linux and OS X - systems. However, you may use any other programming language. As an example, - we provide a \proglang{Python} example in the directory - \IRACEHOME{/examples/python}. % + systems. However, you may use any other programming language. We provide examples written in \proglang{Python}, \proglang{MATLAB} and other languages in \IRACEHOME{/examples/}. % Follow these instructions to adjust the given \code{target-runner} template to your algorithm: \begin{enumerate} @@ -656,11 +646,11 @@ \code{COST=\$(cat \$\{STDOUT\} | grep -e '\^{}[[:space:]]*[+-]\textbackslash{}?[0-9]' | cut -f1)} \end{center} parses the output of your algorithm to obtain the result from the last line. The \code{target-runner} - script must return \textbf{only} one number. In the template example, the result is returned with - \code{echo "\$COST"} (assuming \parameter{maxExperiments} is used) and the used files are deleted. + script must print \textbf{only} one number. In the template example, the result is printed with + \code{echo "\$COST"} (assuming \parameter{maxExperiments} is used) and the generated files are deleted (you may remove that line if you wish to keep them). \begin{xwarningbox} - The \code{target-runner} script must be executable. + The \code{target-runner} script must be an executable file, unless you specify \parameter{targetRunnerLauncher} and \parameter{targetRunnerLauncherArgs}. \end{xwarningbox} You can test the target runner from the \aR console by checking the scenario as @@ -674,7 +664,7 @@ \item \textit{Optional}: Modify the \code{target-evaluator} file. This is rarely needed and the \code{target-runner} template does not use it. \autoref{sec:evaluator} explains when a \parameter{targetEvaluator} is needed and how to define it. - \item The \irace executable provides an option (\parameter{-{}-check}) to + \item The \irace executable provides an option (\parameter{--check}) to check that the scenario is correctly defined. We recommend to perform a check every time you create a new scenario. When performing the check, \irace will verify that the scenario and parameter definitions are @@ -707,7 +697,7 @@ or directly from the \aR console: \begin{itemize} -\item{% +\item \textbf{From the command-line console}, call the command (on Windows, you should execute \code{irace.exe}): \begin{lstlisting}[style=BashInputStyle] @@ -721,8 +711,8 @@ properly in the \code{scenario.txt} file using the options described in \autoref{sec:irace options}. Most \irace options can be specified in the command line or directly in the \code{scenario.txt} file. -} -\item{ + +\item \textbf{From the \aR console}, evaluate: <>= @@ -733,7 +723,7 @@ scenario = defaultScenario()) irace.main(scenario = scenario) @ -} + \end{itemize} This will perform one run of \irace. See the output of \code{irace --help} in the command-line or \code{irace.usage()} in \aR for quick information on @@ -747,17 +737,14 @@ \subsection{Setup example for ACOTSP}\label{sec:example} -The \ACOTSP tuning example can be found in the package installation at - \IRACEHOME{/examples/acotsp}. +The \ACOTSP tuning example can be found in the package installation in the folder \IRACEHOME{/examples/acotsp}. % -Additionally, a number of example scenarios can be found in the \code{examples} folder. More +Other example scenarios can be found in the same folder. More examples of tuning scenarios can be found in the Algorithm Configuration Library (AClib, \url{http://www.aclib.net/}). In this section, we describe how to execute the \ACOTSP scenario. If you wish to start setting up your own scenario, continue to the next section. For this example, we assume -a GNU/Linux system but making the necessary changes in the commands and \parameter{targetRunner}, -it can be executed in any system that has a \proglang{C} compiler. -%\MANUEL{I don't think this is true, since the target-runner script needs Bash} +a GNU/Linux system such as Ubuntu with a working \proglang{C} compiler such as \code{gcc}. To execute this scenario follow these steps: \begin{enumerate} @@ -772,7 +759,7 @@ \end{lstlisting} %@ -\item Download the training instances from \url{http://iridia.ulb.ac.be/irace/} to the \path{~/tuning/} directory. +\item Download the training instances from \url{https://iridia.ulb.ac.be/supp/IridiaSupp2016-003/index.html} to the \path{~/tuning/} directory. \item Create the instance directory (\eg~\path{~/tuning/Instances}) and decompress the instance files on it. %<>= @@ -793,14 +780,6 @@ make \end{lstlisting} %@ -\item Create a directory for the executable and copy it: - -%<>= -\begin{lstlisting}[style=BashInputStyle] -mkdir ~/bin/ -cp ~/tuning/ACOTSP-1.03/acotsp ~/bin/ -\end{lstlisting} -%@ \item Create a directory for executing the experiments and execute \irace: @@ -859,8 +838,7 @@ option \parameter{digits}. For example, given the default number of digits of $4$, the values $0.12345$ and $0.12341$ are both rounded to $0.1234$. - % However, the values $0.00001$ and $0.00005$ remain the same. - Selected real-valued parameters can be optionally sampled on + Selected real-valued parameters can be optionally sampled on a logarithmic scale (base $e$). \item \textit{Integer} parameters are numerical parameters that can take only @@ -896,7 +874,55 @@ defined on them. All fixed parameters must be defined as categorical parameters and have a domain of one element. -\subsubsection{Conditional parameters} +\subsubsection{Parameter dependent domains}\label{sec:paramdependant} + +Domains that are dependent on the values of other parameters can be specified +only for numerical parameters (both integer and real). To do so, the dependent +domain must be expressed in function of another parameter, which must be a +numerical parameter. The expression that defines a dependency must be written +between quotes: \code{(value, "expression")} or \code{("expression", value)} or +\code{("expression", "expression")}. + +The expressions can only use the following operators and \aR functions: \code{+}, +\code{-}, \code{*}, \code{/}, \code{\%\%}, \code{min}, \code{max}, +\code{round}, \code{floor}, \code{ceiling}, \code{trunc}. If you need to use an +operator or function not listed here, please contact us. + +\begin{xwarningbox} +The user must ensure that the defined domain is valid at all times since +\irace currently is not able to detect possible invalid domains based on the expressions +provided. +\end{xwarningbox} + +If you have a parameter \code{p2} that is just a transformation of another +\code{p1}, then instead of using a dependent domain (left-hand side of the +following example), it will be better to create a dummy parameter that controls +the transformation (right-hand side) and do the transformation within \code{target-runner}. For example: +% +\begin{center} +\begin{minipage}{0.4\linewidth} +\small\centering% +\begin{CodeInput}[frame=single] +# With dependent domains +p1 "" r (0, 100) +p2 "" r ("p1", "p1 + 10") +\end{CodeInput} +\end{minipage} +\hspace{\stretch{1}} should be \hspace{\stretch{1}} +\begin{minipage}{0.4\linewidth} +\small\centering% +\begin{CodeInput}[frame=single] +# With a dummy parameter +p1 "" r (0, 100) +p2dum "" r (0, 10) +\end{CodeInput} +\end{minipage} +\end{center} +% +and \code{target-runner} will compute $\code{p2} = \code{p2dum} \cdot \code{p1}$. + + +\subsubsection{Conditional parameters}\label{sec:conditional} Conditional parameters are active only when others have certain values. These dependencies define a hierarchical relation between parameters. For example, @@ -911,39 +937,40 @@ table. Each line of the table defines a configurable parameter % \begin{center} -\code{\