aegis -New_Test(1) General Commands Manual aegis -New_Test(1) NNAAMMEE aegis new test - add a new test to a change SSYYNNOOPPSSIISS aaeeggiiss --NNeeww__TTeesstt [ _o_p_t_i_o_n... ][ _f_i_l_e_n_a_m_e... ] aaeeggiiss --NNeeww__TTeesstt --LLiisstt [ _o_p_t_i_o_n... ] aaeeggiiss --NNeeww__TTeesstt --HHeellpp DDEESSCCRRIIPPTTIIOONN The _a_e_g_i_s _-_N_e_w___T_e_s_t command is used to add a new test to a change. A new file is created in the development directory. New tests default to "automatic" unless otherwise specified. FFiillee NNaammee IInntteerrpprreettaattiioonn The aegis program will attempt to determine the project file names from the file names given on the command line. All file names are stored within aegis projects as relative to the root of the baseline directory tree. The development directory and the integration direc- tory are shadows of this baseline directory, and so these relative names apply here, too. Files named on the command line are first con- verted to absolute paths if necessary. They are then compared with the baseline path, the development directory path, and the integration directory path, to determine a baseline-relative name. It is an error if the file named is outside one of these directory trees. The --BBAAssee__RREEllaattiivvee option may be used to cause relative filenames to be interpreted as relative to the baseline path; absolute filenames will still be compared with the various paths in order to determine a baseline-relative name. The _r_e_l_a_t_i_v_e___f_i_l_e_n_a_m_e___p_r_e_f_e_r_e_n_c_e in the user configuration file may be used to modify this default behavior. See _a_e_u_c_o_n_f(5) for more infor- mation. TTeesstt FFiilleennaammee GGeenneerraattiioonn You may choose your own filename for a test, by specifying it on the command line. If no filename is specified on the command line, a test filename is automatically generated. This is controlled by the _n_e_w___t_e_s_t___f_i_l_e_n_a_m_e field of the project configuration file (see _a_e_p_c_o_n_f(5) for more information. All automatically generated test filenames within a project are numbered uniquely. The default pattern for new test file- names is "_t_e_s_t_/_X_X_/_t_X_X_X_X_[_a_m_]_._s_h", where _X_X is the first 2 digits of the test number, _X_X_X_X is the whole test number, and _[_a_m_] is a for auto- matic tests and m for manual tests. MMooddiiffyyiinngg TTeessttss Tests may be modified in future by adding them to a change with the _a_e_c_p(1) command. Tests are treated just like any other source file, and are subject to the same process. FFiillee TTeemmppllaatteess When a new file is created in the development directory the project _c_o_n_f_i_g file is searched for a template for the new file. If a tem- plate is found, the new file will be initialized to the template, oth- erwise it will be created empty. See _a_e_p_c_o_n_f(5) for more information. The simplest form is to use template files, such as file_template = [ { pattern = [ "*.c" ]; body = "${read_file ${source template/c abs}}"; }, { pattern = [ "test/*/.sh" ]; body = "${read_file ${source template/test abs}}"; }, ]; As you can see, the template files are part of the project source, so you can add the appropriate copyright notices, and wrappers, _e_t_c. The _$_s_o_u_r_c_e substitution locates them, if they are not part of the current change (and they usually are not). The template files themselves contain substitutions. The _$_f_i_l_e_n_a_m_e substitution is available, and contains the name of the file being created. This can be manipulated in various ways when constructing the appropriate file contents. See _a_e_s_u_b(5) for more information about substitutions. It is also possible to run a command to create the new file. You can do this instead of specifying a body string, _v_i_z_: file_template = [ { pattern = [ "*" ]; body_command = "perl ${source template.pl abs} $filename"; }, ]; The command is run with a current directory set to the top of the development directory. It is an error if the command fails to create the file. You can mix-and-match the two techniques, _b_o_d_y string and _b_o_d_y___c_o_m_m_a_n_d, if you want. Be careful to make sure that the test filename template pattern matches the _n_e_w___t_e_s_t___f_i_l_e_n_a_m_e field. FFiillee NNaammee LLiimmiittaattiioonnss There are a number of controls available to limit the form of project file names. All of these controls may be found in the project config- uration file, see _a_e_p_c_o_n_f(5) for more information. The most signifi- cant are briefly described here: maximum_filename_length = integer; This field is used to limit the length of filenames. All new files may not have path components longer than this. Defaults to 255 if not set. For maximum portability you should set this to 14. posix_filename_charset = boolean; This field may be used to limit the characters allowed in filenames to only those explicitly allowed by POSIX. Defaults to _f_a_l_s_e if not set, meaning whatever your operating system will tolerate, except white space and high-bit-on characters. For maximum portability you should set this to _t_r_u_e. dos_filename_required = boolean; This field may be used to limit filenames so that they conform to the DOS 8+3 filename limits and to the DOS filename charac- ter set. Defaults to _f_a_l_s_e if not set. windows_filename_required = boolean; This field may be used to limit filenames so that they conform to the Windows98 and WindowsNT filename limits and character set. Defaults to _f_a_l_s_e if not set. shell_safe_filenames = boolean; This field may be used to limit filenames so that they do not contain shell special characters. Defaults to _t_r_u_e if not set. If this field is set to _f_a_l_s_e, you will need to use the _$_{_q_u_o_t_e_} substitution around filenames in commands, to ensure that filenames containing shell special characters do not have unintended side effects. Weird characters in filenames may also confuse your dependency maintenance tool. allow_white_space_in_filenames = boolean; This field may be used to allow white space characters in file names. This will allow the following characters to appear in file names: backspace (BS, \b, 0x08), horizontal tab (HT, \t, 0x09), new line (NL, \n, 0x0A), vertical tab (VT, \v, 0x0B), form feed (FF, \f, 0x0C), and carriage return (CR, \r, 0x0D). Defaults to false if not set. Note that this field does not override other file name fil- ters. It will be necessary to explicitly set _s_h_e_l_l___s_a_f_e___- _f_i_l_e_n_a_m_e_s _= _f_a_l_s_e as well. It will be necessary to set _d_o_s___- _f_i_l_e_n_a_m_e___r_e_q_u_i_r_e_d _= _f_a_l_s_e (the default) as well. It will be necessary to set _p_o_s_i_x___f_i_l_e_n_a_m_e___c_h_a_r_s_e_t _= _f_a_l_s_e (the default) as well. The user must take great care to use the ${quote} substitution around all file names in commands in the project configura- tion. And even then, substitutions which expect a space sepa- rated list of file names will have undefined results. allow_non_ascii_filenames = boolean; This field may be used to allow file names with non-ascii- printable characters in them. Usually this would mean a UTF8 or international charset of some kind. Defaults to false if not set. Note that this field does not override other file name fil- ters. It will be necessary to explicitly set _s_h_e_l_l___s_a_f_e___- _f_i_l_e_n_a_m_e_s _= _f_a_l_s_e as well. It will be necessary to set _d_o_s___- _f_i_l_e_n_a_m_e___r_e_q_u_i_r_e_d _= _f_a_l_s_e (the default) as well. It will be necessary to set _p_o_s_i_x___f_i_l_e_n_a_m_e___c_h_a_r_s_e_t _= _f_a_l_s_e (the default) as well. filename_pattern_accept = [ string ]; This field is used to specify a list of patterns of acceptable filenames. Defaults to "*" if not set. filename_pattern_reject = [ string ]; This field is used to specify a list of patterns of unaccept- able filenames. PPlleeaassee NNoottee:: Aegis also consults the underlying file system, to deter- mine its notion of maximum file size. Where the file system's maximum file size is less than _m_a_x_i_m_u_m___f_i_l_e_n_a_m_e___l_e_n_g_t_h, the filesystem wins. This can happen, for example, when you are using the Linux UMSDOS file system, or when you have an NFS mounted an ancient V7 filesystem. Setting _m_a_x_i_m_u_m___f_i_l_e_n_a_m_e___l_e_n_g_t_h to 255 in these cases does not alter the fact that the underlying file systems limits are far smaller (12 and 14, respectively). If your development directories (or your whole project) is on filesys- tems with filename limitations, or a portion of the heterogeneous builds take place in such an environment, it helps to tell Aegis what they are (using the project _c_o_n_f_i_g file's fields) so that you don't run into the situation where the project builds on the more permissive environments, but fails with mysterious errors in the more limited environments. If your development directories are routinely on a Linux UMSDOS filesystem, you would probably be better off setting _d_o_s___f_i_l_e_- _n_a_m_e___r_e_q_u_i_r_e_d _= _t_r_u_e, and also changing the _d_e_v_e_l_o_p_m_e_n_t___d_i_r_e_c_t_o_r_y___t_e_m_- _p_l_a_t_e field. Heterogeneous development with various Windows environ- ments may also require this. CChhaannggiinngg tthhee TTyyppee ooff aa FFiillee If you want to change the type of a file (say, from a test to a source file, or _v_i_c_e _v_e_r_s_a) you could do it as two changes, by first using _a_e_r_m(1) in one change and then using _a_e_n_f(1) or _a_e_n_t(1) in a second change, or you can combine both steps in the same change. Remember to use the _a_e_r_m _-_n_o_w_h_i_t_e_o_u_t option or you will get a most peculiar new file template. NNoottiiffiiccaattiioonn The _n_e_w___t_e_s_t___c_o_m_m_a_n_d in the project _c_o_n_f_i_g file is run, if set. The _p_r_o_j_e_c_t___f_i_l_e___c_o_m_m_a_n_d is also run, if set, and if there has been an integration recently. See _a_e_p_c_o_n_f(5) for more information. TTEESSTT PPRROOCCEESSSS Each change is required to be accompanied by tests, and those tests are required to be run against the built development directory, and they must pass. This ensures that new functionality is accompanied by tests to verify its correctness, and bug fixes are accompanied by tests which confirm that the bug has been fixed. RReeggrreessssiioonn TTeessttss Tests are treated as any other source file, and are maintained in the baseline and history with all other source files. The tests which must accompany every change accumulate in the project baseline, pro- viding a definition of correct function for the baseline. These accu- mulated tests may be executed using an "aegis -REGression" command, to verify that the project will not "regress" as a result of a change. BBaasseelliinnee TTeessttss Bug fixes are required to have their tests _f_a_i_l against the project baseline (in contrast to the development directory). This ensures that the test actually demonstrates the bug in the baseline, as well as demonstrating that it is fixed by the change. New functionality trivially fails against the baseline, and so aegis does not attempt to guess if a test is a bug fix test or new functionality test, it simply requires tests to fail against the baseline. This requirement applies both to new tests being created by a change and also to tests which have been copied into a change for modifica- tion. RReevviieewwiinngg TTeessttss Reviewers may be confident that aegis has enforced the test require- ments; that a change must have tests, that the change must build, that the tests pass against the development directory, and that the tests fail against the baseline. These conditions are enforced by _a_e_d_e(1) and the change will not be advanced to the _b_e_i_n_g _r_e_v_i_e_w_e_d state until these conditions are met. Reviewers should thus review tests for _c_o_m_- _p_l_e_t_e_n_e_s_s of coverage of the code in the change, and insensitivity to changes in the execution environment (e.g. not date sensitive). Reviewers should also use "aegis -list change_details" to verify that a change does or does not have testing exemptions. EExxeemmppttiioonnss Various test exemptions may be granted by project administrators, see _a_e_p_a(1) and _a_e_p_a_t_t_r(5) for more information. Copying tests into a change, or adding new tests to a change, may cancel those exemptions. TTEESSTT CCOORRRREELLAATTIIOONNSS The "aegis -Test -SUGgest" command may be used to have aegis suggest suitable regression tests for your change, based on the source files in your change. This automatically focuses testing effort to relevant tests, reducing the number of regression tests necessary to be confi- dent that you have not introduced a bug. The test correlations are generated by the "aegis -Integrate_Pass" command, which associates each test in the change with each source file in the change. Thus, each source file accumulates a list of tests which have been associated with it in the past. This is not as exact as code coverage analysis, but is a reasonable approximation in practice. The _a_e_c_p(1) and _a_e_n_f(1) commands are used to associate files with a change. While they do not actively perform the association, these are the files used by _a_e_i_p_a_s_s(1) and _a_e_t(1) to determine which source files are associated with which tests. TTeesstt CCoorrrreellaattiioonn AAccccuurraaccyy Assuming that the testing correlations are accurate and that the tests are evenly distributed across the function space, there will be a less than _1_/_n_u_m_b_e_r chance that a relevant test has not been run by the "aegis -Test -SUGgest _n_u_m_b_e_r" command. A small amount of noise is added to the test weighting, so that unexpected things are sometimes tested, and the same tests are not run every time. Test correlation accuracy can be improved by ensuring that: +o Each change should be strongly focused, with no gratuitous file inclusions. This avoids spurious correlations. +o Each item of new functionality should be added in an individual change, rather than several together. This strongly correlates tests with functionality. +o Each bug should be fixed in an individual change, rather than sev- eral together. This strongly correlates tests with functionality. +o Test correlations will be lost if files are moved. This is because correlations are by name. The best way for tests to correlate accurately with source files is when a change contains a test and exactly those files relating to the functionality under test. Too many spurious files will weaken the usefulness of the testing correlations. OOPPTTIIOONNSS The following options are understood; --AAUUTTOOmmaattiicc This option may be used to specify automatic tests. Automatic tests require no human assistance. --BBAAssee__RREEllaattiivvee This option may be used to cause relative filenames to be con- sidered relative to the base of the source tree. See _a_e_u_- _c_o_n_f(5) for the corresponding user preference. --CCUUrrrreenntt__RREEllaattiivvee This option may be used to cause relative filenames to be con- sidered relative to the current directory. This is usually the default. See _a_e_u_c_o_n_f(5) for the corresponding user pref- erence. --CChhaannggee _n_u_m_b_e_r This option may be used to specify a particular change within a project. See _a_e_g_i_s(1) for a complete description of this option. --EEddiitt Edit the new test files one they have been created. (This avoids the copy-and-paste step required to edit the new test script when it has an automatically generated file name.) --HHeellpp This option may be used to obtain more information about how to use the _a_e_g_i_s program. --LLiisstt This option may be used to obtain a list of suitable subjects for this command. The list may be more general than expected. --MMAANNuuaall This option may be used to specify manual tests. Manual tests require some human intervention, e.g.: confirmation of some screen behavior (X11, for instance), or some user action, "unplug ethernet cable now". --NNoott__LLooggggiinngg This option may be used to disable the automatic logging of output and errors to a file. This is often useful when sev- eral aegis commands are combined in a shell script. --OOuuttppuutt _f_i_l_e_n_a_m_e This option may be used to specify a filename which is to be written with the automatically determined test file name. Useful for writing scripts. --PPrroojjeecctt _n_a_m_e This option may be used to select the project of interest. When no --PPrroojjeecctt option is specified, the _A_E_G_I_S___P_R_O_J_E_C_T envi- ronment variable is consulted. If that does not exist, the user's _$_H_O_M_E_/_._a_e_g_i_s_r_c file is examined for a default project field (see _a_e_u_c_o_n_f(5) for more information). If that does not exist, when the user is only working on changes within a sin- gle project, the project name defaults to that project. Oth- erwise, it is an error. --TTEEMMppllaattee This option may be used to specify that a new file template should be used, even if the file already exists. --NNoo__TTEEMMppllaattee This option may be used to specify that a new file template should not be used, even if the file does not exist (any empty file will be created). --TTEERRssee This option may be used to cause listings to produce the bare minimum of information. It is usually useful for shell scripts. --UUnniivveerrssaall__UUnniiqquuee__IIDDeennttiiffiieerr _s_t_r_i_n_g This option may be used to set the UUID of a file. --NNoott__UUnniivveerrssaall__UUnniiqquuee__IIDDeennttiiffiieerr This option may be used to require that the file is created without an UUID. The aaeeiippaassss--ooppttiioonn::aassssiiggnn--ffiillee--uuuuiidd is set to false for the file to avoid automatic UUID assignment when aaeeiippaassss(1) is invoked. --VVeerrbboossee This option may be used to cause aegis to produce more output. By default aegis only produces output on errors. When used with the --LLiisstt option this option causes column headings to be added. --WWaaiitt This option may be used to require Aegis commands to wait for access locks, if they cannot be obtained immediately. Defaults to the user's _l_o_c_k___w_a_i_t___p_r_e_f_e_r_e_n_c_e if not specified, see _a_e_u_c_o_n_f(5) for more information. --NNoo__WWaaiitt This option may be used to require Aegis commands to emit a fatal error if access locks cannot be obtained immediately. Defaults to the user's _l_o_c_k___w_a_i_t___p_r_e_f_e_r_e_n_c_e if not specified, see _a_e_u_c_o_n_f(5) for more information. See also _a_e_g_i_s(1) for options common to all aegis commands. All options may be abbreviated; the abbreviation is documented as the upper case letters, all lower case letters and underscores (_) are optional. You must use consecutive sequences of optional letters. All options are case insensitive, you may type them in upper case or lower case or a combination of both, case is not important. For example: the arguments "-project", "-PROJ" and "-p" are all inter- preted to mean the --PPrroojjeecctt option. The argument "-prj" will not be understood, because consecutive optional characters were not supplied. Options and other command line arguments may be mixed arbitrarily on the command line, after the function selectors. The GNU long option names are understood. Since all option names for _a_e_g_i_s are long, this means ignoring the extra leading '-'. The "----_o_p_t_i_o_n==_v_a_l_u_e" convention is also understood. RREECCOOMMMMEENNDDEEDD AALLIIAASS The recommended alias for this command is csh% alias aent 'aegis -nt \!* -v' sh$ aent(){aegis -nt "$@" -v} EERRRROORRSS It is an error if the change is not in the _b_e_i_n_g _d_e_v_e_l_o_p_e_d state. It is an error if the change is not assigned to the current user. EEXXIITT SSTTAATTUUSS The _a_e_g_i_s command will exit with a status of 1 on any error. The _a_e_g_i_s command will only exit with a status of 0 if there are no errors. EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS See _a_e_g_i_s(1) for a list of environment variables which may affect this command. See _a_e_p_c_o_n_f(5) for the project configuration file's _p_r_o_j_e_c_t___s_p_e_c_i_f_i_c field for how to set environment variables for all commands executed by Aegis. SSEEEE AALLSSOO _a_e_c_p(1) copy an existing test into a change _a_e_d_b(1) begin development of a change _a_e_n_t_u(1) remove a new test from a change _a_e_r_m(1) remove an existing test as part of a change _a_e_t(1) run tests _a_e_u_c_o_n_f(5) user configuration file format CCOOPPYYRRIIGGHHTT aegis version 4.25.D611 Copyright (C) 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Peter Miller The aegis program comes with ABSOLUTELY NO WARRANTY; for details use the '_a_e_g_i_s _-_V_E_R_S_i_o_n _L_i_c_e_n_s_e' command. This is free software and you are welcome to redistribute it under certain conditions; for details use the '_a_e_g_i_s _-_V_E_R_S_i_o_n _L_i_c_e_n_s_e' command. AAUUTTHHOORR Peter Miller E-Mail: pmiller@opensource.org.au /\/\* WWW: http://miller.emu.id.au/pmiller/ Reference Manual Aegis aegis -New_Test(1)