aeintegratq(1) General Commands Manual aeintegratq(1) NNAAMMEE aeintegratq - integrate changes into projects SSYYNNOOPPSSIISS aaeeiinntteeggrraattqq [ _o_p_t_i_o_n... ] _p_r_o_j_e_c_t_-_n_a_m_e... DDEESSCCRRIIPPTTIIOONN The _a_e_i_n_t_e_g_r_a_t_q command is used to manage the integrations of one or more changes in one or more projects. Normally run via _c_r_o_n(1) or _a_t(1) with the name of a single project, aeintegratq will manage all operations for integration even when --BBuuiilldd and --TTeesstt are required on multiple architectures. If a change review is revoked after the queue is running aeintegratq will notice the bad state and silently move on. If one or more changes are ended or passed after the queue is running, and -loop has been given, aeintegratq will notice the new change[s] and integrate them. Additional options allow the integrator full con- trol over most aspects of queue management such as the order of inte- gration of multiple changes. OOPPTTIIOONNSS The following options are understood: OOppttiioonn SSuummmmaarryy --hh Help, show usage information. --HH Help, show usage plus all helpful comment information. --aa run on Any machine (normally only IntegrationHost) --ss run remote operations via ssh (default rsh) --nn No action, just tell what would be done. --iibb _s Specify (remote) server for ibegin. --iipp _s Specify (remote) server for ipass. --kk Keep the scripts and report files. --KK Keep the temp file even if integration passes. --lloooopp Loop to process more changes if they become available before aeintegratq completes. It will stop when there is nothing more to be done. --MM _l_i_s_t Minimum, run given changes _-_m_i_n_i_m_u_m --PP _l_i_s_t Precious, do not IIFFaaiill changes in _l_i_s_t, just stop. --RR _l_i_s_t Ready, specify order and subset, _e_._g_. -R 29,45 --SS _s_t_a_g_e Pick up at given stage (diff|build|test|integrate) --cc _c_h_a_n_g_e_-_n_u_m_b_e_r specify Change to integrate at Stage --pp _p_r_o_j_e_c_t_-_n_a_m_e specify single Project name NOTE: if custom options such as -P -R -S -c -p are given only a single project may be integrated since the options would be meaningless to the next project given. Some options are present only for testing and investigation. Note that options are rarely required for normal operations. CCoonnttrrooll OOppttiioonnss The following options are available for special needs. They control the order and disposition of each change aawwaaiittiinngg__iinntteeggrraattiioonn in a given project. --RR[eady] _n_u_m_b_e_r_1_,_n_u_m_b_e_r_2_._._. This option is used to specify order or subset to integrate. Only those changes listed will be attempted, and in exactly the order given. This applies to queue looping if --lloooopp is given. In particular note unless the list includes future changes, future loops will not integrate them. Useful if a particular change must go in before another for some reason. Or if only integrating one or two changes when several are aawwaaiittiinngg__iinntteeggrraattiioonn in the given project. A sin- gle change may also be specified with the --cc[[hhaannggee]] _n_u_m_b_e_r option, which is common for other aegis commands. However the --RR option allows a list and if given will override any --cc given. --PP[recious] _n_u_m_b_e_r_1_,_n_u_m_b_e_r_2_._._. --PP[recious] aallll This option is used to specify that a particular change or subset of changes should be considered pprreecciioouuss. It neither implies order nor limits the queue run to that subset; it only means that the changes should be considered pprreecciioouuss. Note that at least one number (or the keyword _a_l_l) must be given. The concept of pprreecciioouuss means that if the given change were to fail anywhere in the integration process, then the process simply stops and leaves the problem change in the delta direc- tory. The --IIFFaaiill would not actually be executed. This is sometimes useful to diagnose a problem which only occurs dur- ing integrations. It is also useful if the failure is due to a transient problem such as unreliable machines on the net- work. In such a case the integration can be resumed after fixing the problem. See the _s_t_a_g_e options below. If, on the other hand, a pprreecciioouuss change makes it through the integration process successfully, the option has no effect. --MM[inimum] _n_u_m_b_e_r_1_,_n_u_m_b_e_r_2_._._. or _a_l_l Integrate the given change[s] with the --mmiinniimmuumm option. Such changes will be put on the end of the queue so that the last integrations of a run will be a minimum. This feature allows practical use of minimum integrations without requiring --mmiinnii-- mmuumm on each and every integration. See the section below on _M_i_n_i_m_u_m _i_n_t_e_g_r_a_t_i_o_n_s for more information. If --lloooopp is given any change[s] specified as minimum will run at the end of the loop in which they are ready, they will not be pushed to the final loop. --iibb[server] _s_e_r_v_e_r_-_n_a_m_e or "" --iipp[server] _s_e_r_v_e_r_-_n_a_m_e or "" To specify a remote server on which to run --iibbeeggiinn or 00 rreessppeecciivveellyy.. TThheessee ooppttiioonnss aarree rraarreellyy nneeeeddeedd,, bbuutt mmaayy bbee uussee-- ffuull iiff aa pprroojjeecctt iiss hhoosstteedd oonn aa ddiiffffeerreenntt ffiillee sseerrvveerr aanndd hhaass aa llaarrggee bbaasseelliinnee.. BByy hhaavviinngg tthhee --iibbeeggiinn rruunn oonn tthhaatt sseerrvveerr tthhee nneettwwoorrkk ttrraaffffiicc wwoouulldd bbee ggrreeaattllyy rreedduucceedd aanndd ffoorr llaarrggee pprroojjeeccttss aanndd//oorr ssllooww nneettwwoorrkkss ccaann ggrreeaattllyy rreedduuccee tthhee ttiimmee rreeqquuiirreedd ffoorr --iibbeeggiinn.. TThhee ooppttiioonn ffoorrmm ooff ggiivviinngg aann eemmppttyy nnaammee ddeeppeennddss oonn tthhee oouuttppuutt ooff ddff --kk ggiivviinngg aa ppaarrsseeaabbllee hhoosstt nnaammee.. IIff tthhaatt iiss nnoott ttrruuee oonn yyoouurr iinntteeggrraattiioonn hhoosstt aarrcchhiitteeccttuurree,, yyoouu wwiillll hhaavvee ttoo ssppeecciiffyy tthhee sseerrvveerr nnaammee.. --ddiissppllaayy _d_i_s_p_l_a_y_-_v_a_l_u_e oorr """" To specify a valid X display for use during integration opera- tions. SSttaaggee OOppttiioonnss The following options allow [re]starting an integration which has already progressed through some stages. This is useful to deal with failed (_p_r_e_c_i_o_u_s) integrations, or to finish automatically an integra- tion begun by hand. --SS[tage] ddiiffff --SS[tage] bbuuiilldd --SS[tage] tteesstt --SS[tage] iinntteeggrraattee Pick up the integration at the given ssttaaggee. Requires --cc[hange] _n_u_m_b_e_r option to specify the change number. AAddvvaanncceedd CCoonnttrroollss The integrator may provide for special situations such as operations required after --BBuuiilldd and before --TTeesstt, or at the end of a queue run. Such capabilities are provided by hhooookkss and ssttrraatteeggiieess described below. HHooookkss There are a set of _h_o_o_k_s available which are run, if present, before and after each stage of the integration. They can be used to help ensure that the integrator actually gets some sleep while managing large projects. These hooks are searched for in the directory $$HHOOMMEE//iinntteeggrraattiioonn__hhooookkss. None need exist; aeintegratq will only pay attention to any that do exist. Hooks may be any form of executable (script, etc) and are called with 2 arguments: pprroojjeecctt--nnaammee cchhaannggee--nnuummbbeerr. They run as the integrator on the machine from which aeintegratq was started. They are named using the project name along with a suffix according to what place in the integration process you want them to run. Note that if a hook for project ffoooo exists it is also used for any branches under that project. For example, if you have provided ffoooo..pprree__iipp, it will be run for foo.1 and foo.1.0 as well. If for some reason you want different (or no) action for project ffoooo..11..00, then you would provide ffoooo..11..00..pprree__iipp which does what you wish, including noth- ing, effectively overriding ffoooo..pprree__iipp. Here is how to map particular places in the integration process to hook suffixes. +--------------------------------------------------+ |run at time extension | +--------------------------------------------------+ |before attempting -Integrate_Begin .pre_ib | |after -Integrate_Begin completes .ib | |before attempting -Diff .pre_d | |after -Diff completes .d | |before attempting -Build .pre_b | |after -Build completes .b | |before attempting -Build on .pre_b | |after -Build on completes .b | |before attempting -Test .pre_t | |after -Test completes .t | |before attempting -IPass .pre_ip | |after -IPass completes .ip | |before attempting -IFail .pre_if | |after -IFail completes .if | +--------------------------------------------------+ The hook program should exit with 0 if successful or 1 if not. A non- zero exit causes the change being integrated to fail immediately unless it was marked precious. Note that in most cases anything done via an ..iipp hook should probably be done instead by the _i_p_a_s_s___n_o_t_i_f_y command in the project attributes file (see _a_e_p_a_t_t_r(5) for more information), or the _b_u_i_l_d___t_i_m_e___a_d_j_u_s_t___- _n_o_t_i_f_y___c_o_m_m_a_n_d in the project configuration file (see _a_e_p_c_o_n_f(5) for more information), but the hook can provide a temporary way to keep going until the permanent solution can be implemented. In addition two special hooks, aaeeiinntteeggrraattqq..eenndd and aaeeiinntteeggrraattqq..ffaaiill, are recognized. They are called when aaeeiinntteeggrraattqq finishes a queue run. They are called with 2 arguments like any other hook (pprroojjeecctt-- nnaammee cchhaannggee--nnuummbbeerr) although both the project-name and change-number given are of the last change integrated and may be less than useful. The ..eenndd hook is called if/when the queue run is finished and was suc- cessful. Note that this does not mean that no changes failed, only that no queue errors occurred. This hook might be used to invoke another queue run on a different project/branch, or possibly even on the same project, if other changes may have been ended and/or reviewed while the first run was in progress, see also the --lloooopp option. These conditions arise quite often with flex time engineers. Another use of the ..eenndd hook is to automatically build a new package using the newly integrated project as source. If queue errors were encountered, or a change failed that was marked _p_r_e_c_i_o_u_s, then the ..ffaaiill hook is called. An obvious use of that hook would be an e-mailed page to the integrator. SSttrraatteeggyy oorr OOooppss--rreettrryy Sometimes a persistent build problem will plague integrations. This can be very annoying if it ruins an overnight run, especially if the cure is simple when it happens. Examples of this can be timeouts due to a busy data server or other transient errors. Note that this applies only to --BBuuiilldd related problems. To deal with such problems the integrator may provide a _s_t_r_a_t_e_g_y script specific to a project. An executable program should be found in $$HHOOMMEE//ssttrraatteeggyy..<>. The program will be run as the integrator with the _d_e_l_t_a directory as current directory. The program may do any commands necessary to clean up and/or diagnose the error. If the script finds the problem to be transient and fix-able, it exits successfully (with 0 status) and aeintegratq will re-launch the --BBuuiilldd and log the re-try. Otherwise the script should exit with a 1 and the change will fail. MMuullttii--AArrcchhiitteeccttuurree iinntteeggrraattiioonnss For projects which build and test on multiple architectures, aeinte- gratq requires _a_r_c_h___h_o_s_t_s be installed and have available at least one machine of each architecture required. This is also true if the host from which aeintegratq is run is of a different architecture from the target architecture of the project being integrated. If you wish to take advantage of multiple architecture automatic inte- grations, you can install _a_r_c_h___h_o_s_t_s or provide a more simple script which will return a machine name according to architecture and job type. MMiinniimmuumm iinntteeggrraattiioonnss provides a mmiinniimmuumm integration capability which may be used for vari- ous reasons. The term mmiinniimmuumm may be a bit counter intuitive. One might think it means to do the mmiinniimmuumm amount of work, however it actually means use a mmiinniimmuumm of files from the baseline in populating the _d_e_l_t_a directory. Since no constructed files are put in the _d_e_l_t_a directory, this normally leads to actually building everything in the project from sources and, as such, might be considered the most robust of builds. Note that any change which removes a file, whether by _a_e_r_m or _a_e_m_v, results in an implicit mmiinniimmuumm integration. This is intended to ensure nothing in the project references the removed file. A project may adopt a policy that a product release should be based on a minimum integration. Such a policy may be a reflection of local confidence, or lack thereof, in the project's DMT (Dependency Mainte- nance Tool) or build system. Or it may be based on a validation process wishing to make a simple statement on how the released package was produced. Another, more transient, reason a to require a minimum integration might be when upgrading a third party library, compiler or maybe even OS level. Any of these events would signal the need for a minimum integration to ensure everything is rebuilt using the new resources. This can be done with minimum overhead using the --MM option as described above. The cost of a mmiinniimmuumm integration varies according to type and size of the project. For very large projects, especially those building large numbers of binaries, the cost can be large. However large projects also require significant time to fully populate the delta directory. A minimum integration only copies those files under aegis control, skipping all "produced" files. In the case where a file upon which everything depends is changed, everything will be built anyway so the copy of the already built files is a waste of time. This means that sometimes a minimum can be as cheap as a normal integration. MMaannuuaall TTeessttss allows tests to be defined as _m_a_n_u_a_l which may be necessary if the test requires human interaction or some transient resource. Such tests can be problematic for automatic integrations and generally must have some means to pass without running during integrations. For this, and other, reasons most sites seek to avoid _m_a_n_u_a_l tests. There are a num- ber of ways to code a test such that it will pass automatically during integrations. Just one example for shell script tests might be: CSTATE=`aesub -p $AEGIS_PROJECT -c $AEGIS_CHANGE '${state}'` if [ "$CSTATE" = "being_integrated" ] then echo "`basename $0` passes during integration" exit 0 fi OOppttiioonnaall SSuuppppoorrtt PPrrooggrraammss There are some programs which aeintegratq will use if they are installed. +o _a_r_c_h___h_o_s_t_s was mentioned previously. It is optional only if your projects and your file server are of a single architecture. +o _a_e_l_o_g_r_e_s may enhance the information provided in _-_I_F_a_i_l entries. Normally all you get is the last 10 lines of the log file, which is not bad if tests fail, but can be terrible for failed builds. If you provide a program named _a_e_l_o_g_r_e_s which knows how to extract a better succinct report of problems, the output of that program will be used instead of the simple tail. It is called with a _-_i option. +o ssoouunndd__aallll__mmaacchhiinneess, if available, will be called when integrations either pass or fail. It can be helpful to announce the fact that an integration has finished. If it passed, developers will probably want to do an aaeedd to bring their changes up to date. The audio announce- ment provides another timely hint. The sound files are searched for in the /usr/local/com/aegis/sounds directory. They will have endings of ___p_a_s_s and ___f_a_i_l according to the results of a given attempt. Two sound files are required: _i_n_t_e_g_r_a_- _t_i_o_n___p_a_s_s and _i_n_t_e_g_r_a_t_i_o_n___f_a_i_l. Others will be used if provided to customize the sounds so that each developer may have one or more per- sonal sounds. If a file named _<_d_e_v_e_l_o_p_e_r_>___p_a_s_s is located, it will be used. If a set of files exist named _<_d_e_v_e_l_o_p_e_r___p_a_s_s_._[_0_-_9_] they will be used in random sequence. The same search rule applies to ___f_a_i_l sets. The _s_o_u_n_d___a_l_l___m_a_c_h_i_n_e_s program may use a host list and play the sound file on each machine or, assuming that other audio capabilities exist, might do any form of announcement desired. EEXXIITT SSTTAATTUUSS The _a_e_i_n_t_e_g_r_a_t_q command will exit with a status of 1 on any error. The _a_e_i_n_t_e_g_r_a_t_q command will only exit with a status of 0 if there are no errors. EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS See _a_e_g_i_s(1) for a list of environment variables which may affect this command. See _a_e_p_c_o_n_f(5) for the project configuration file's _p_r_o_j_e_c_t___s_p_e_c_i_f_i_c field for how to set environment variables for all commands executed by Aegis. FFIILLEESS Control files are searched for in the _$_H_O_M_E directory. They are named strategy., They need not exist if no special action is neces- sary. The hook scripts are searched for in the _$_H_O_M_E_/_i_n_t_e_g_r_a_t_i_o_n___h_o_o_k_s directory. They are named .. Also _a_e_i_n_t_e_g_r_a_t_q_._e_n_d and _a_e_i_n_t_e_g_r_a_t_q_._f_a_i_l. These hooks also need not exist if no special action is desired. CCOOPPYYRRIIGGHHTT aeintegratq version 4.25.D611 Copyright (C) 1998-2005 Endocardial Solutions, Inc. The aeintegratq program comes with ABSOLUTELY NO WARRANTY; This is free software and you are welcome to redistribute it under certain conditions; Reference Manual Aegis aeintegratq(1)