Reliability of HROs and sensemaking approaches

Reliability from the sensemaking approach

18The essence of sensemaking is to provide an understanding of how meaning and artifacts are produced and reproduced collectively (Czarniawska-Joerges, 1992; Saleh, et al., 2010). Communication is a key component in this process (Weick, Sutcliffe & Obstfeld, 2005). It “involves turning circumstances into a situation that is comprehended explicitly in words and that serves as a springboard into action” (Weick, et al., 2005: 409). In practice, actors constantly engage in sensemaking of their environments in high-complexity and high-turbulence environments (Boudes & Laroche, 2009; Brown, 2004). In these situations, their ability “in shifting from centralized authority during routine operations to local/ decentralized authority for hazardous situations” (Saleh, et al., 2010: 1110) requires that these communications be integrated in an organizational culture centered on reliability. The latter allows for the creation of “a homogeneous set of assumptions and decision premises which, when they are invoked on a local and decentralized basis, preserve coordination and centralization” (Weick, 1987: 124).

19The quality of communication in an organizational culture of reliability is thus a necessary condition for organizational reliability (Kœnig, et al., 2016), especially when these organizations are confronted with contexts that combine, at the same time, an information overload with constant turbulence and increasing complexity (Weick, 1995).

20However, in CTI situations, the sensemaking approach faces issues that hard to solve because of its core principles. Indeed, in CTI, the strict time constraints limit the sense of building efficiency in the interactions— sensemaking (Weick, 1995, 2001)—and a learning process by trial/error is not possible because of risks of critical irreversibility (Weick, 1987; Weick & Sutcliffe, 2011). Moreover, as shown above, the strategies used to manage information overload (omission, error tolerance, filtering, abstraction to attenuate the information overload) do not suit CTI situations, either because these strategies allow errors in the decision-making process or because they accept simplified considerations (Sutcliffe & Weick, 2006). In CTI situations, decision-makers may find themselves in situations that are subjectively complex in which the risk of leaving their comfort zone justifies the necessity to interact in order to have the same perception of the situation (Weick, 1993; Yami, Chappert & Mione, 2015).

21So, the issues for a decision-maker are how to quickly generate valid knowledge knowing that: 1) deliberating requires time; 2) CTI situations require rapid action; and 3) in case of error, there is a risk of irreversible damage to the entity being managed.

22Thus, in CTI situations, both the sensemaking and the HRO approaches face difficulties that are hard to resolve given the principles that characterize them. Table 1 summarizes these difficulties.

Table 1

Issues difficult to resolve in HRO and sensemaking approaches

Issues for a decision-maker to solve in the context of CTI situations using the:
HRO approach	Sensemaking approach
When a decision-maker is confronted with a new situation and knows that he/she has to act quickly and that in case of error there is a risk of irreversible damage to the entity being managed, he/she must consider: - How to solve conflicts between rules in real time? - How to re-prioritize a set of rules in real time?	How to quickly generate valid knowledge knowing that: 1) deliberating requires time 2) CTI situations require quick action 3) in case of error, there is a risk of irreversible damage to the entity being managed? How in practice can a decision-maker manage a situation of information overload knowing that: 1) CTI situations require quick action and 2) in case of error there is a risk of irreversible damage to the entity being managed?

Issues difficult to resolve in HRO and sensemaking approaches

Reliability from a meta rules perspective

23In CTI situations, the high risk of irreversibility and the requirement to act quickly mean that a key issue for decision-makers is to preserve the integrity of the controlled system, which is a necessary condition for achieving the objectives. We propose introducing the concept of meta rules in view of increased reliability.

24The concept of meta rules has appeared in different research fields, such as education, business research and entrepreneurship (Fuller, Argyle & Moran, 2009). At first sight, meta rules correspond to one form of meta-level knowledge. Davis (1980) defines them as rules governing a set of lower-level rules that constitute a framework of rules for which the priorities might change. In CTI situations, the primary function of meta rules is to identify critical structural points that enable decision-makers to gain an overview of the managed entity (e.g., service, organization) in order to preserve its integrity (i.e., to maintain the vital functions of the system). The central issue is to avoid decision errors that might lead to a degradation of the system’s capacity to achieve its objectives (Bradley, Easley & Stolle, 2001; Brady, 1987; Davis, 1980).

25Thus, from a conceptual point of view, meta rules fundamentally concern the sustainability of the entity being managed. That is why rules and meta rules cannot be considered on the same level. In this setting, meta rules entail more than identifying a limited number of simple rules (Cazenave, 2003; Cox, 2005; Davis, 1980). With meta rules, it is possible to attain an overall view of a situation, allowing decision-makers to redefine their priorities if necessary and reallocate resources to ensure the integrity of the controlled system. When meta rules reflect a logic of preservation and sustainability of the entity being managed, rules focus above all on solving the problem in the most efficient way.

26In addition, meta rules must be differentiated from heuristics, as analyzed by Bingham and Eisenhardt (2011), for whom heuristics refer to specific problem contexts and focus on capturing opportunities in dynamic environments. Moreover, they occur with restricted information, which is not necessarily the case for meta rules. In contrast, meta rules should enable decision-makers to construct a big picture, with good situation awareness [1] (Endsley, 2001). They form an integrative framework that supports quick decision-making while limiting erroneous choices.

27Thus, the concept of meta rules should allow rapid action (as a response to the constraint of “time stress”) by making reliable decisions (as a response to the constraint risk of “critical irreversibility”) in CTI situations.

Reliability, rules and meta rules

28Our argument about the relationship between rules and meta rules is organized around three propositions.

29Proposition 1: As already stated, a strict compliance with rules plays a central role in the reliability of HROs (Roberts & Rousseau, 1989). Rules enable the decision-maker to decide quickly about issues which have already been covered. According to the HRO approach, the compliance with rules is a key reliability factor, leading us to formulate the following proposition:

30P1: Complying with rules has a positive effect on the reliability level.

31Proposition 2: The reliability level associated with Proposition 1 can be improved by the use of meta rules. Indeed, in a real situation, the decision-maker faces not one but several problems. The issue “problem encountered/rule to be mobilized” must, therefore, be re-evaluated quickly in view of the other problems that the decision-maker faces simultaneously. Thus, meta rules, because they enable the decision-maker to have a global intelligence of the entity being managed, are likely to play the role of a mediating variable between the rules and the reliability. These elements lead us to consider that the mobilization of a larger number of rules, due to the multiplicity of problems to be solved, should imply for the decision-maker a more frequent use of meta rules. The use of meta rules should improve the reliability of decision-making. This leads us to formulate the following proposition:

32P2: Meta rules play a mediating role between the rules and reliability, which can be decomposed into two effects: (a) the mobilization of a more important number of rules - due to the multiplicity of problems to be solved—is associated with a more important use of meta rules (first part of the mediating effect); (b) meta rules—because they make an overall understanding of the situation possible—have a positive effect on the reliability level (second part of the mediating effect).

33Proposition 3: Situations with increasing complexity are, however, more difficult for the decision-maker to solve with rules-based behavior. Indeed, in these situations, the identification and the choice of the rules to be followed take time, all the more so if the decision-maker does not know all the rules. He/she has to check the written documents detailing the rules that apply for the situation. In an HRO context, the complexity of technologies requires that all procedures are precisely written (Roberts & Rousseau, 1989). But reading these documents takes time, whereas the decision-maker is facing CTI situation constraints.

34These elements lead us to consider that, in situations with increasing complexity, the decision- maker should mobilize fewer and fewer rules in favor of a more important use of meta rules. Indeed, only meta rules enable the decision-maker to quickly identify the critical points on which he/she must act to ensure the integrity of the entity being managed. As the use of meta rules is positively associated with reliability, this leads us to formulate the following complementary proposition:

35P3: In situations with increasing complexity, meta rules play a mediating role between the rules and reliability, which can be decomposed into two effects: (a) less use of rules—due to increasing complexity—is associated with a more important use of meta rules (first part of the mediating effect of meta rules); (b) meta rules—that are concerned with preserving the integrity of the entity to be managed—have a positive effect on the reliability level (second part of the mediating effect of meta rules).

Figure 1

Reliability, rules, meta rules and complexity (Model 1)

Reliability, rules, meta rules and complexity (Model 1)

Reliability, interactions and meta rules

36Our argument about the relationship between interactions and meta rules is constructed, as previously, around three propositions.

37Proposition 4: In unusual or unknown situations, the socio-constructionist approach of sensemaking (Weick, et al., 2005) considers the interactions as a means of generating new knowledge. The interactions enable the decision-maker to have an understanding of the situation. They avoid the strategies of “blind search” (Rasmussen, 1983) and can be a condition for organizational reliability (Weick, 1995). These elements lead us to propose the following proposition:

38P4: Interactions positively affect reliability.

39Proposition 5: The reliability level associated with Proposition 4 can be improved by the use of meta rules. The difficulty with the knowledge produced by the interactions is that this knowledge is not necessarily “robust” (Artinger, Petersen, Gigerenzer & Weibler, 2015; Maitland & Sammartino, 2015). Some researchers go beyond this constraint and highlight that the use of simple rules or “frugal heuristics” (Artinger, et al., 2015; Goldstein & Gigerenzer, 2009) can be a rational strategy (Bingham & Eisenhardt, 2011). They enable a degree of plasticity—the ability to improvise— (Davis, et al., 2009) while providing a framework that “keeps behaviors at least partially coherent” (Bingham & Eisenhardt, 2011). However, as noted before, CTI situations require not only rapid action but also no errors in the decision-making (Faraj & Xiao, 2006; Roberts, 1990) due to risks of critical irreversibility.

40These elements lead us to consider that, in CTI situations, the mobilization of meta rules by the decision-maker should enable him/her to analyze rapidly whether what is produced by interactions in the sensemaking process is suitable or not. Indeed, meta rules allow for a rapidly global understanding of the sensitive points for the entity being managed and ensure that relevant decisions are taken to maintain the integrity of the entity. Thus, the relationship between interactions and the use of meta rules should be positive, and the use of these meta rules should positively affect reliability. These elements lead us to propose the following proposition:

41P5: Meta rules play a mediating role between interactions and reliability, which are composed of two effects: (a) the important mobilization of interactions, due to the multiplicity of problems to be solved, is associated with a greater use of meta rules (first part of the mediating effect); (b) the meta rules, as noted, have a positive effect on level of reliability (second part of the mediating effect).

42Proposition 6: However, in situations with increasing complexity, it becomes more difficult for the decision-maker to interact with his/her teammates. Indeed, in these situations, the deliberation process requires more time to gain intelligence of the situation, which is hardly compatible with the constraints of CTI situations. As Rasmussen (1983: 261) points out about human capabilities and knowledge-based behaviors, “only a few elements of a problem can be within the span of attention simultaneously. This means that the complex net of causal relations of an environment must be treated in a chain of mental operations”.

43These elements lead us to consider that in CTI situations with increasing complexity there is no simple linear relation between the multiplicity of interactions and the sense produced; more (fewer) interactions do not produce more (less) sense. One way to unravel this “limit of human capabilities” (Rasmussen, 1983: 264) is to change the level of abstraction. In a situation of increasing complexity, it is more a question of mobilizing meta rules than resorting to more interactions. The mobilization of the latter remains, however, without simple linear relations with the level of interactions. These remarks lead us to formulate the following proposition:

44P6: The incidence of level of increasing complexity on the mediating effect of meta rules between the interactions and reliability is different between the first and the second part of this mediating effect; thus (a) due to the limits of human cognitive capabilities, there is no simple linear relationship—positive or negative—between the interactions and the meta rules as the level of complexity increases. In other words, more complexity does not necessarily involve more interactions in its relationship with meta rules (first part of the mediating effect of meta rules); (b) in increasing complexity, the use of meta rules—as explained previously—has a positive effect on level of reliability (second part of the mediating effect of meta rules).

Figure 2

Reliability, interactions, meta rules and complexity (Model 2):

Reliability, interactions, meta rules and complexity (Model 2):

Reliability and comparison “rules” vs “interactions”

45Proposition 7: Finally, we propose comparing the explanatory power of the HRO and sensemaking approaches.

46By taking into account the specificities of CTI situations, which make the learning process by trial/error impossible due to emergency constraints and irreversible decisions in case of error, the association of rules— identified as the source of reliability (Weick, 1993)—with the flexibility in real time of decision-making through the use of meta rules, should allow for a higher level of reliability to be obtained. In other words, the association of the HRO approach with meta rules should have, in the specific case of CTI situations, a better explanatory power than the explanatory power from the association of meta rules with interactions.

47Hence, we further formulated the following proposal:

48P7: The overall reliability level is higher with the HRO approach (Model 1) than with the sensemaking approach (Model 2).

Field of study

49To test our propositions, we chose the French Navy, which corresponds to the features of HRO, (Roberts, et al., 1994) and specifically the French Naval Academy. The Naval Academy trains French Navy career officers whose responsibilities include guiding and commanding operational units. During their training at the Naval Academy, officers develop the skills that will make them the future leaders of the navy. The theoretical and practical training takes place over six semesters and focuses on three fundamental areas: maritime training, humanities and military training and scientific training. After graduation from the academy, career officers will occupy different positions, depending on their chosen training, whether in operations (e.g., underwater warfare, surface warfare, naval aviation, commando) or operational support (e.g., propulsion, aeronautical and nuclear energy). Later in their careers, officers often take more strategic roles and occupy positions in the general staff, in planning or expert roles, or leading in the fields of operations, weaponry, intelligence or human resources. They may command one or several operational units (e.g., commando, aircraft fleet, frigate, submarine, naval base).

50Of the three fundamental training areas, maritime training involves the progressive acquisition of seamanship, knowledge and practice in maneuvering and navigating, as well as an understanding of the propulsion and energy production systems used on board ships. This training provides cadets with the skills necessary to perform the roles of an officer, operations officer or engineering officer of the watch. Their education is provided by professional sailors. Regular trips, several weeks in length, are scheduled throughout the instructional program, and the cadets’ skills are tested during a long-term mission at sea in their sixth semester. The practical application takes place on instructional ships (traditional and racing sailboats, motor boats, training ships) and with a navigation simulator recreating a full-scale navigational bridge at scale 1 surrounded by 360° screens depicting a maritime environment.

Sample design

51The Naval Academy simulator consists of four bridges and a command post that allow users to practice various navigation situations based on different scenarios. The primary goal of the simulator is to provide students with the basic skills of an officer of the watch on board a ship before they navigate the actual training ship. This simulator can also monitor cadets’ reactions to complex or dangerous situations in a virtual environment. Therefore, we selected this simulator for this study.

52To create the scenarios on the simulator, we used a database corresponding to accidents that occurred during real nautical situations on board navy vessels. After analyzing this database, we retained the situations in which the conditions of complexity, urgency and human errors led to irreversible damage (death) in order to work on CTI situation criteria. We used the features of these critical situations to transform them into scenarios on the simulator—one scenario per team of three cadets based on the same kind of constraints: the proximity to danger, problems of visibility and technical problems. While each scenario was based on the same situations, the scenarios for each team differed slightly in the order in which the constraints appeared and the geographical location (Brest Harbour or Lorient Harbour). The scenario reflected real nautical events and particularly complex situations with critical points such as:

53

• a navigation situation (a moving boat)
• a low level of visibility
• the emergence of a new element in a short period of time (other boats, fire)
• dangers close by (rocks, coast, floating buoy).

54The conditions of complexity arose from the necessity to manage different new elements and the rules associated with them, while taking account of the nearby dangers and the rules that had to be followed to deal with them. The time pressure conditions arose from the short period of time available to manage all these parameters. In such conditions, a situation becomes increasingly complex due to the sequence of problems to be solved in a short period of time (Halford, Wilson & Phillips, 1998). And lastly, the error irreversibility conditions came from the serious potential consequences to the vessel (damage to the boat hull, its equipment and even to crew members), with possible effects on the integrity of the system and its sustainability. To illustrate this work, in Table 2, we present the scenario “Arrival at Lorient Harbour” that six different teams had to manage:

Table 2

Scenario “Arrival at Lorient Harbour”

	Arrival at Lorient Harbour – examples of six team scenarios
	Team 1 on bridge A	Team 2 on bridge B	Team 3 on bridge C
1st 120’ session	Bridge A 9h47: 1 inflatable boat right in front of the ship + 1 tugboat 9h52: 1 interruption by a phone contact from Lorient Harbour 9h52: 1 tugboat 9h58: 1 sailboat + marking buoy to be avoided	Bridge B 9h38: 1 marking buoy to avoid 9h39: 1 tugboat very close on the starboard side 9h42: 1 training ship 9h43: 1 interruption by a phone contact from Lorient Harbour 9h44: 1 inflatable boat	Bridge C 9h22: 1 tugboat + 1 floating tree trunk 9h29: interruption by a phone contact from Lorient Harbour 9h29: 1 inflatable boat 9h31: 1 inflatable boat 9h30: 2nd training ship
	Team 4 on bridge A	Team 5 on bridge B	Team 6 on bridge C
2nd 120’ session	11h25: 1 fishing trawler 11h25: 1 gyro breakdown 11h27: 1 man overboard 11h28: 1 VIP on the bridge	11h25: 1 military ship & 1 fishing trawler 11h26: 1 interruption 11h27: 1 breakdown & 1 boat 11h28: 1 water leak	11h41: 1 military ship 11h43: 1 gyro breakdown at 5° + 1 boat on the port side and 2 marking buoys 11h4: a journalist and a VIP on the bridge 11h53: 1 gyro breakdown + 1 boat on the port side 11h56: gyro OK 11h58: 1 inflatable boat on the starboard side

Scenario “Arrival at Lorient Harbour”

Data collection

55As previously explained, the simulator was used as research equipment to implement the nautical scenario and to observe the teams’ behaviors during the scenario; they were recorded on a questionnaire created specifically for this test and filled out by experts (instructors).

56Our sample was made up of 108 bridge teams of three cadets, each playing three different roles, namely the leader (F1), the deputy officer of the watch (F2) and the watchkeeper (F3). As only 274 cadets are trained at the French Naval Academy each year, some played several team member roles on several teams.

57Each team had a nautical mission on a simulator bridge (place unit) lasting 120 minutes (time unit) with a period of variability (with complexity, time pressure, risk of error) of five to ten minutes. The CTI situations were created from a command control room. Indeed, both the inputs and events came from a central command room.

58One expert completed a questionnaire for each bridge team, so that the number of questionnaires corresponded to the number of teams (108). We tested this questionnaire in advance, using a verbal protocol method and a focus group with 10 experts (i.e., instructors assessing cadets on the simulator).

59During these two-hour navigation periods, to create CTI conditions, technicians working in the central command room inserted variability periods of five to ten minutes corresponding to unexpected situations, with a series of events based on real nautical situations recreating a complex, urgent situation with risk of severe damage. Indeed, each scenario contained a 5-to-10-minute period whose complexity appeared through the introduction of multiple incidents, difficult sea conditions and time pressure. The details of our work on the simulator were based on that of Journé (Journé, 1999; Journé & Raulet-Croset, 2008).

60The questionnaire allowed the 10 instructors to identify compliance with the rules, the level of interactions between the three team members and the use of meta rules. The objective was to identify the associations of variables that allow teams to achieve the highest level of reliability.

Variables

61Our observations focused on five variables: compliance with rules, interactions, use of meta rules, reliability level and complexity level.

62Our propositions are based on the association of these variables. In this section, we explain how we operationalized each of these variables with measurement scales. The specific measurement scales developed for this work are based on prior literature, expert analyses and semi-structured interviews with French Navy instructors.

• Independent Variables:

Compliance with Rules

63According to Romelaer (1998), a formal rule can be considered as a regularity of behavior constituting a formalized standard expressed with items. The works of Crozier and Friedberg (1977) and Weick (1998) emphasize the possible behaviors of actors regarding compliance with rules: actors can respect them, respect them while maintaining a margin for maneuver or deviate from them by improvising.

64Measurement scales of procedures are uncommon. Bunn’s (1994) work on the measurement of procedural control, based on ten items on a seven-point Likert scale, focuses more on the relevance of procedures than on their use. As this seemed to be less suitable to our research objective, we proposed the measurement of the variable “rules,” given the specificity of the field of study.

65Indeed, compliance with rules is formally included in the document, General Guidelines for Boating Safety (2012). These rules correspond to the detailed rules of nautical safety which are available for crews on “clipboards” on the bridge. They are included in a list of rules to be applied to each type of nautical situation. Therefore, expert assessors can determine if the rules have been strictly respected or if any kind of deviation has occurred.

66We, therefore, proposed that the “rules” variable be measured on a one-dimensional scale relative to the compliance with rules by the bridge team members. The rule-based actions are specified by each rule (Davis, et al., 2009). To explain the differences between rules and meta rules, we give illustrations from our studied field; rules represent rules written on a clipboard available on the bridge for bridge teams. These rules are written and give instructions on how to take specific action in a given situation (for example, rules for “man overboard”, including a sequence of orders to be strictly respected, described in Table 3).

Table 3

Rules to be followed for a “man overboard” situation

Function for the leader F1
1/	Identify the location of the man overboard on GPS (button MARK/MOB)
2/	Warn the ship crew by pressing the horn 4 times and saying, “a man overboard at X location, the teams responsible for rescue boat at your position. This is not an exercise”
3/	Begin the maneuver corresponding to the plan
4/	“Operator’s maneuver in the machine room for man overboard rule”
5/	Communications VHF on P3 (mobile VHF) for the rescue boat and for the bridge switched on and tested
6/	The rescue boat is prepared for action
7/	Stop the fire pump
8/	Give the order “launch the rescue boat”
9/	Indicate the location of the man overboard to the rescue boat
Coordinate actions with F2, the deputy officer of the watch, and F3, the watchkeeper

Rules to be followed for a “man overboard” situation

67Thus, for a warship, we measured the compliance with rules by using a one-dimensional scale with three items, measured on a five-point Likert scale that indicates each actor’s compliance with the rules in the nautical situation. Instructors are expert assessors, they can determine if the rules have been strictly respected. The measurement of the rules variable is specified in Appendix 1.

Interactions

68We chose the “interactions” variable because this variable is one of the four sources of resilience identified by Weick (1993). Moreover, the importance of this variable also emerges from the results of our quantitative study based on the analysis of accidents (Navy database), underlining that lack of communication constitutes a major source of error.

69While Weick (1993), when he identifies “respectful interactions” as a source of resilience, gives it a larger meaning, we focused on the verbal aspect of interactions.

70We measured the “interactions” variable from the verbal interactions between the members of the bridge team (between F1 and deputy officer F2, between F1 and the other members of the team, i.e., F3, between F2 and F3) using a questionnaire designed to measure this variable. The measurement of the interactions variable is specified in Appendix 1.

• Mediating variable

Meta rules

71Meta rules enable actors to prioritize some of the rules and, therefore, arbitrate among them (Davis, Buchanan & Shortliffe, 1977). These meta rules enable actors to gain an abstract overview of the system and thus quickly diagnose its level of control. For our investigation, the item “applying rules of a general level” and “applying rules of a higher level” corresponded to five meta rules of general navigation principles, as presented at the conference on the General Guidelines for Conduct at Sea (DG NAUT, 2012) and summarized in Table 4.

72We differentiated these meta rules from rules. If rule-based actions are specified by each rule (Davis, et al., 2009), meta rules are not specific to one type of action in a situation. Davis (1980) defines meta rules as rules governing a set of lower-level rules, corresponding to one form of meta-level knowledge. That is why “to call for watchfulness”, “to reduce one’s speed”, “to take visual cues in the environment”, “to request an exemption from the captain for one action (not provided by rules)”, and “to report to the captain changes in the nautical situation” (in order to request an exemption) have been identified by experts (ten instructors in a focus group) as meta rules. They correspond to global measures of rules and enable the decision-maker (the leader of the bridge team) to take, the first time, measures suitable for any kind of nautical situation (corresponding to the features of meta rules) in order to preserve the integrity of the system and its capacity to carry out the mission and make it possible, the second time, to respect detailed rules for specific situations. The meta rules described in Table 4 were identified as meta rules by experts because, for any kind of situation, they enable the crew members to create a framework for actions to preserve the integrity of the ship.

Table 4

Meta rules for the officer on watch

• to call for watchfulness

• to reduce one’s speed

• to take visual cues in the environment

• to request an exemption from the captain for one action (not provided by rules)

• to report to the captain changes in the nautical situation (in order to request an exemption)

Meta rules for the officer on watch

73We chose the meta rules resulting from the focus group as the measurement criteria for this variable. The measurement of the meta rules variable is specified in Appendix 1.

Statistical technique

78For this study, we used the partial least square (PLS) structural equations model technique based on variances. This method is appropriate because it has a number of advantages related to its better adaptation to exploratory research (Tenenhaus, Vinzi, Chatelin & Lauro, 2005) and it is a technique particularly suitable for incorporating composite constructs (Chin, 1998; Haenlein & Kaplan, 2004). For this research, the rules and meta rules variables were regarded as composite constructs, while the reliability and interaction constructs were defined as factors.

79In order to analyze the mediating effect of the meta rule variable moderated by complexity, the methodology of analysis of indirect effects was applied (Zhao, Lynch & Chen, 2010). Version 2.12.2 of the macro PROCESS for IBMSPSS Statistics 20 (Hayes, 2017) was used to perform a post hoc conditional process analysis to examine whether the indirect effect of the rule construct on reliability in Model l (interactions on reliability in Model 2) through the meta rule construct was conditioned/moderated by complexity. To test the proposals, we used bootstrap analysis, with 5,000 subsamples being carried out for all the empirical analyses (Leal-Rodríguez, Roldán, Ariza-Montes & Leal-Millán, 2014).

Measure validation of the models

80We ensured the content validity of the constructs by adapting previously validated scales. The reliability and convergent validity of the reflective constructs (reliability and interactions) were evaluated as follows (Henseler, Hubona & Ray, 2016). Construct reliability was evaluated by checking whether the construct composite reliability (Dijstra & Henseler’s rho (pA)) and indicator loadings were above the threshold of 0.71 (Chin, 2010). Convergent validity was tested by examining whether the construct average variance extracted (AVE) was above the recommended value of 0.5 (Hair, Black & Babin, 2010) The reflective construct discriminant validity in Model 2 was tested by analyzing whether the square root of the construct AVE was vertical and horizontally greater than its correlation with other constructs (Gefen, Straub & Boudreau, 2000).

81Composite constructs (rule and meta rule constructs) should be assessed differently to reflective constructs since traditional assessments of reliability and validity do not apply well to formative constructs (Petter, Straub & Rai, 2007). Multicollinearity, weights and their level of significance have to be checked (Benitez & Ray, 2012; Cenfetelli & Bassellier, 2009). We checked for multicollinearity by estimating the variance inflation factor (VIF) for the three rules indicators, three interactions construct indicators and four meta rules indicators. All these VIF values were lower than 10, suggesting that multicollinearity was not a problem in our data (Petter, et al., 2007). Only one item linked to the meta rule construct had a non-significant positive weight of 0.236 (Model 1) and 0.033 (Model 2). As the factor loadings were significant and contributed highly to the meta rule concept, we decided to keep it (Benitez, Henseler & Castillo, 2017). Overall, this analysis suggested good properties for all the constructs (Appendices 1 and 2).

Results

82The results for Model 1 (Table 5) show that rules have a positive and very significant relationship with reliability (beta: 0.744, t-value: 11.272). This result supports P1. Model 1 also reveals that rules have a positive and significant relationship with meta rules (beta: 0.587, t-value: 9.074), which in turn enabled the team to improve reliability (beta: 0.145, t-value: 2.066).

Table 5

Structural model evaluation: Model 1 rule, meta rule and reliability

Model 1	Beta (t-test, p-value)
Rule → Reliability	0.744 (11.272; 0.000)
Rule → Meta rule	0.587 (9.074; 0.000)
Meta rule → Reliability	0.145 (2.066; 0.019)
Complexity → Meta rule	0.200 (2.543; 0.06)
Complexity → Reliability	0.072 (1.440; 0.075)
Complexity Rule → Meta rule	- 0.072 (1.580; 0.057)
Complexity Meta rule →Reliability	0.067 (1.448; 0.074)
Indirect effects
Rule-reliability through meta rule Confidence intervals	0.085 (1.787; 0.037) [0.017-0.174]
Note: t-values are presented in parentheses. Confidence intervals are presented in brackets. Adjusted R2 (Meta rule: 0.403; Reliability: 0.739). SRMR: 0.058.
Post hoc conditional process analysis. Indirect effect of rule over reliability through meta rule conditioned to complexity
Complexity	Indirect effects	Boot standard error	Confidence intervals
- 1.468	0.040	0.065	[-0.046-0.215]
- 0.404	0.075	0.048	[0.001-0.184]
- 0.404	0.075	0.048	[0.001-0.184]
0.660	0.111	0.051	[0.021-0.217]
1.724	0.147	0.079	[0.019-0.329]
Values for complexity are 10th, 25th, 50th, 75th, and 90th percentiles. Level of confidence for all confidence intervals 95. Level of confidence for all confidence intervals 95

Structural model evaluation: Model 1 rule, meta rule and reliability

83The indirect effect of rules over reliability by means of meta rules was tested using the indirect effects analysis methodology proposed by Zhao et al. (2010). Therefore, we developed a complementary mediation analysis by performing a bootstrap confidence interval test which enabled us to measure the total and specific indirect effects of the intermediate variable included in Model 1 (Preacher & Hayes, 2004). The results show that the indirect effect of rules on reliability is significant. This is why it is evident that this effect was mediated by the meta rules construct, which supports P2.

84But the question to answer is whether this indirect effect differs depending on the complexity of a situation. The interaction effect between rules and complexity had a negative relationship with meta rules (beta: - 0.072, t-value: 1.440), while the interaction effect between meta rules and complexity had a positive relationship with reliability (beta: 0.067, t-value: 1.448). Overall, complexity moderated the indirect effect of rule on reliability through meta rule, as shown by the post hoc conditional process analysis to examine whether the indirect effect of rule on reliability through meta rule was moderated by complexity. We estimated the conditional indirect effects and the bias-corrected 95 % bootstrap confidence intervals for the indirect effects at various values of complexity. A confidence interval not including zero means that the indirect effect of rule is significant at a specific value of complexity. Complexity was estimated as the 10th, 25th, 50th, 75th and 90th percentiles of the distribution in the sample (Leal-Rodríguez, et al., 2014). Consequently, this result suggests that rule affects reliability by means of meta rule, and that this indirect effect is stronger when the complexity is high (P3).

Table 6

Structural model evaluation: Model 2 Interactions, meta rule and reliability

Model 2	Beta (t-value)
Interactions → Reliability	0.514 (5.889)
Interactions → Meta rule	0.544 (7.301)
Meta rule → Reliability	0.332 (3.629)
Complexity → Meta rule	0.157 (1.693)
Complexity → Reliability	0.069 (1.300)
Complexity Interactions → Meta rule	- 0.035 (0.560)
Complexity Meta rule → Reliability	0.077 (1.496)
Note: t-values are presented in parentheses. Confidence intervals are presented in brackets. Adjusted R2 (Meta rule: 0.325; Reliability: 0.595). SRMR: 0.071.
Indirect effects
Interaction-reliability through meta rule Confidence Intervals	0.181 (2.729) [0.100-0.316]
Post hoc conditional process analysis. Indirect effect of interaction variable over reliability through meta rule conditioned to complexity
Complexity	Indirect effects	Boot standard error	Confidence intervals
- 1.468	0.133	0.115	[-0.004-0.442]
- 0.404	0.169	0.079	[0.059-0.352]
- 0.404	0.169	0.079	[0.059-0.352]
0.660	0.207	0.069	[0.081-0.348]
1.724	0.244	0.118	[0.070-0.541]
Values for complexity are 10th, 25th, 50th, 75th, and 90th percentiles. Level of confidence for all confidence intervals 95. Level of confidence for all confidence intervals 95.

Structural model evaluation: Model 2 Interactions, meta rule and reliability

85The estimates reveal that the interactions construct had a positive relationship with reliability (beta: 0.514, t-value: 5.889), verifying P4. The model also discloses that interactions had a positive and significant relationship with meta rule (beta: 0.544, t-value: 7.301), which in turn allowed the team to improve reliability (beta: 0.332, t-value: 3.629). In this model, the indirect effect of the interactions construct on reliability through meta rule was also verified (beta: 0.181, t-value: 2.729). Therefore, meta rule mediates the effect of the interactions construct on reliability and increases it, which supports P5. This indirect effect was moderated by the complexity level as the interaction effect between meta rule and complexity affects reliability (beta: 0.077, t-value: 1.496).

86The interaction effect between the interactions construct and complexity did not affect meta rule (beta: -0.035, t-value: 0.560). This result was reinforced in the post hoc process analysis as the confidence intervals for the indirect effect when complexity increased did not include the value 0. At low level of complexity, this variable did not affect the indirect effect of interactions on reliability, but when complexity was 0.660 the confidence interval did not include 0 [0.081-0.348], so the indirect effect was conditioned by the level of complexity. Consequently, P6 can be verified because there is evidence to support a moderated effect of complexity on the relationship between meta rule and reliability but not on the relationship between interaction and meta rule.

87When comparing the R² for Model 1 and Model 2, the HRO approach clearly shows a higher capacity to predict reliability (adjusted R² : 73.9 %) than the sensemaking approach (adjusted R² : 59.5 %). This result supports P7.

Table 7

Propositions summary and results

Propositions	Results
P1: Complying with rules has a positive effect on the reliability level.	Supported
P2: Meta rules play a mediating role between the rules and the reliability, which can be decomposed into two effects: (a) the mobilization of a more important number of rules—due to the multiplicity of problems to be solved—is associated with a more important use of meta rules (first part of the mediating effect); (b) meta rules—because they make an overall understanding of the situation possible—have a positive effect on the reliability level (second part of the mediating effect).	Supported
P3: In situations with increasing complexity, meta rules play a mediating role between the rules and the reliability, which can be decomposed into two effects: (a) less use of rules—due to increasing complexity—is associated with a more important use of meta rules (first part of the mediating effect of meta rules); (b) meta rules—that are concerned with preserving the integrity of the entity to be managed—have a positive effect on the reliability level (second part of the mediating effect of meta rules).	Supported: complexity moderates the relationship between rule and meta rule, but with a negative coefficient (a) but the total indirect effect is affected by complexity as it affects relationship between meta rule and reliability in a very positive way (b)
P4: Interactions positively affect reliability	Supported
P5: Meta rules play a mediating role between interactions and reliability which are composed of two effects: (a) the important mobilization of interactions, due to the multiplicity of problems to be solved, is associated with a greater use of meta rules (first part of the mediating effect), (b) the meta rules, as noted, have a positive effect on level of reliability (second part of the mediating effect).	Supported
P6: The incidence of level of increasing complexity on the mediating effect of meta rules between the interactions and reliability is different between the first and the second part of this mediating effect; thus (a): due to the limits of human cognitive capabilities, there is no simple linear relationship—positive or negative—between the interactions and the meta rules as the level of complexity increases. In other words, more complexity does not necessarily involve more interactions in its relationship with meta rules (first part of the mediating effect of meta rules). (b): In increasing complexity, the use of meta rules—as explained previously— has a positive effect on level of reliability (second part of the mediating effect of meta rules)	Supported: complexity does not moderate the relationship between interactions and meta rule (a) but the total indirect effect is affected by complexity as it affects relationship between meta rule and reliability (b)
P7: The overall reliability level is higher with the HRO approach (Model 1) than with the sensemaking approach (Model 2).	Supported

Propositions summary and results

Lessons from the “rules–meta rules” model

89The results for Model 1 (Table 5) show that compliance with rules has a positive and very significant relationship with reliability, underlining the relevance of the HRO approach. The results also show that compliance with rules affects reliability by means of meta rules, this indirect effect being stronger when the complexity is high. Indeed, in a situation of increasing complexity, the understanding of a situation requires more and more time. This constraint can be overcome by meta rules that can quickly replace the use of rules in a holistic vision of the entity being managed. These results underline the stake of meta rules when the level of complexity increases.

90The results also teach us that the level of complexity moderates the relationship between compliance with rules and meta rules—with a negative coefficient—underlining a probable substitution process between rules and meta rules in favor of the latest when complexity is increased. In the same way, as expected with an increasing level of complexity, the indirect effect of meta rules becomes stronger, underlining the key role played by meta rules when the level of complexity increases.

91These results mean that when the level of complexity increases in a situation, the actor should refer to meta rules if he/she faces difficulties (no time, rules conflict) in complying with rules.

92However, it should be pointed out that the move from a register of actions mobilizing rules to a register of actions giving a higher priority to meta rules is not obvious. Indeed, on the one hand, the actors may not necessarily be aware of the meta rules because they are trained first of all to respect and not to deviate from the rules, in particular in military institutions, which is the field of study of our article. On the other hand, when there are multiple rules, especially in HRO, knowing them and applying them, particularly in situations of increasing complexity, can gradually mobilize all the cognitive capacities of the decision-maker, leaving little space for the attention that he/she should continue to pay to the global vision of the entity being managed.

93Finally, even if decision-makers are aware of the need to change the action register by referring more to the meta rules, they risk being hampered in this because they know that, by referring less to the rules, they will engage their personal responsibilities if a major and irreversible incident occurs.

94In spite of these difficulties, the results lead us to think that referring to meta rules is a possible solution for helping individuals quickly and easily resolve potential conflicts between different rules and make possible a rehierarchization of rules in real time. Above all, the results remind us that respect for the rules appears to be a better condition for obtaining a higher reliability level, according to the HRO approach. Moreover, the mediating role of meta rules between the rules and the reliability level is a key point that deserves to be researched further.

Lessons from the model “interactions—meta rules” model

95In this second model, the results (Table 6) show that interactions have a positive effect on the reliability level, which also confirms the relevance of the sensemaking approach. However, because of a greater risk of error, the reliability level with interactions is lower than the reliability level when relying solely on rules, underlining that the HRO approach enables actors to obtain a higher level of reliability compared with the sensemaking approach in CTI situations. Moreover, when the level of complexity increases, the overall mediating effect in Model 2 has a positive and significant effect on reliability, showing the relevance of the use of meta rules. These results lead us to conclude that, in Model 2, the mediating role of meta rules positively affects the level of reliability when the complexity increases.

96These findings provide us with a greater understanding of the issues inherent in the improvement of decision-making in CTI situations. First, they highlight that, in the focal study of CTI, the use of interactions is not sufficient to produce robust knowledge for high reliability, as suggested by Weick (1993). Second, the use of meta rules makes it possible to improve the level of reliability. Thus, by offering an integrating framework, it is possible with meta rules to reduce the time of deliberation, as well as the risks of information overload. Moreover, meta rules provide elements that answer the two issues encountered within the framework of the sensemaking approach (shown in Table 1). Lastly, results also show that the overall reliability level is higher with the HRO approach (Model 1) than with the sensemaking approach (Model 2), emphasizing that the better of these two options is to use rules with meta rules. These results underline the value of meta rules for the management of CTI situations.

Conceptual and practical issues pertaining to meta rules

97The use of meta rules is a relevant practice for CTI situations, considering the difficulty associated with addressing all the information related to a sequence of challenging problems in a short period of time, especially if the decision-maker is not an expert. In practice, finding a solution is difficult because the time needed to conduct an analysis exceeds the time available in the situation, which demands rapid decision-making.

98Meta rules, therefore, make it possible to solve some issues with which the HRO and sensemaking approaches are confronted (see Table 1 in section 2). Thus, with the HRO approach, meta rules—because they provide a global vision of the entity to be managed and points of vulnerability—allow a rapid response to potential conflicts between rules and the need or not to re-prioritize them. Similarly, with the sensemaking approach, meta rules allow the meaning produced by the interactions to be replaced with an overall vision of the entity to be managed. This rapid reassessment and this reframing of the meaning produced in the interaction limits, in CTI situations, the risks of information overload.

99The usefulness of meta rules is not limited to CTI situations. Decision-making can be difficult in any situation that characterizes many rules and requires substantial knowledge to take action. Meta rules constitute a facilitator strategy, making it possible to return to the basics and using only the relevant rules. Fundamentally, they make sensemaking possible. Moreover, two main factors justify their ability to lead actors to make robust and reliable decisions: they facilitate the creation of both “a big picture” and “situation awareness” (Endsley, 2001). Thus, meta rules enable decision-makers to stand back temporarily from the pre-established decomposition of tasks (i.e., compliance with rules) and focus on sensemaking. For the decision-maker, understanding the situation through meta rules, at a key time and up to its likely evolution, limits the risk of information overload by facilitating an overall understanding of the situation. It becomes possible to decide quickly, if necessary, which new strategies will be robust and reliable.

100As the advantages of meta rules are likely to go beyond CTI situations, they may offer sources of flexibility and reliability in organizations. They can help to resolve different types of paradoxes, such as “routine versus adaptation”, “structure versus agency” (Feldman & Pentland, 2003), “empowerment versus control” (Jarzabkowski, Lê & Van de Ven, 2013) or “learning versus performing” (Garcias, Dalmasso & Sardas, 2015). Metaphorically, meta rules constitute an “abstract grammar” that enables actors to keep the whole system in mind and thereby identify critical factors for its integrity.