Introduction
The job satisfaction of healthcare professionals is widely concerned, for its impact on physician outcomes (eg, turnover, performance and mental health) and healthcare outcomes (eg, quality of care, patient outcomes and costs).1–4 Hoppock defined job satisfaction as any combination of psychological, physiological and environmental conditions that encourage employees to be satisfied or happy with their job.5 Different personal attributes and working environments which consist of job characteristics, physical working conditions and social working conditions, may affect workers’ job satisfaction.6 As for healthcare professionals, aside from personal factors such as age, gender, marital status, position and education background, abundant evidence suggests that work environment factors significantly influence job satisfaction. These factors include income, working shifts, leadership quality, job autonomy and collegial support.7–11
Health policies have been considered as a type of practice environment which influence the job satisfaction of healthcare professionals.3 12 Prior literature has proven that the impact of hospital mergers on staff job satisfaction and psychological status in the National Health Service.13 14 The Performance Appraisal for Tertiary Public Hospitals (PATPH) in China was initially launched in 2019. It annually evaluated and rated more than 2400 top hospitals which were funded by the government based on five dimensions: medical quality of the hospital, operation efficiency of the hospital, sustainable development of the hospital, satisfaction of patients and job satisfaction of healthcare professionals. The PATPH seeks to promote transitions in both the developmental and the managerial approaches of tertiary public hospitals towards more efficient and quality-driven paradigms.15 So far, the performance appraisals conducted under the PATPH serve as a crucial determinant for various facets of policy formulation, including government support for hospital development, financial allocations, remuneration for healthcare professionals based on performance and hospital revenue.
Throughout the implementation of PATPH, hospitals have been the recipients of initiatives aimed at enhancing efficiency and quality. The provision of financial and non-financial incentives, which are propelled by PATPH, has motivated tertiary public hospitals to identify and address weaknesses across various appraisal dimensions, thereby improving overall performance. This process has led to significant reforms in working environments and social settings within hospitals through the implementation of adaptive policies.12 16 Hence, healthcare professionals have been directly impacted by these adaptive policies at the hospital level.
This study aimed to investigate the impact of PATPH as an environmental factor on the job satisfaction of healthcare professionals. There were ambiguous results about the impact of health policies on the job satisfaction of healthcare professionals. Resulting from alterations in job arrangements, changes in everyday work activities and organisational culture may lead to heightened stress, decreased job security, and reduced job autonomy among healthcare professionals due to stringent regulations and increased requirements.17–19 However, contrary perspectives20 have highlighted that a strong sense of satisfaction stems primarily from internal values rather than external changes. Furthermore, they argue that healthcare reforms have not shown any significant or persistent impact on doctors’ job satisfaction.
At the individual level, healthcare professionals may experience a combination of support and pressure during the implementation of PATPH. On the one hand, PATPH aims to enhance medical quality, improve hospital operational efficiency, promote sustainable hospital development and ensure patient satisfaction, thereby offering favourable working conditions to satisfy healthcare professionals. On the other hand, healthcare professionals may also face more demanding work tasks and higher work requirements due to the pressure from annual hospital rankings, which are linked to governmental funding. For instance, the emphasis on improving the quality of health records and medical care as encouraged by PATPH may place additional stress on healthcare professionals and potentially diminish their job satisfaction.21–23 In such case, it becomes essential to conduct further research on the impact of macro health policies like PATPH, which primarily target organisational changes rather than healthcare professionals, on the well-being of healthcare professionals and their job satisfaction.
Methods
Study design and population
This quantitative study used data sourced from a nationwide cross-sectional multistage sampling survey conducted in tertiary public general hospitals across China in August 2020. According to the economic development level and geographic region, we divided 23 provinces and 5 autonomous regions of China into eastern, central and western regions. In the first stage, we randomly selected one provincial administrative region from each region. In the second stage, to ensure the representativeness among hospitals with different performance levels, one tertiary public hospital was randomly selected as a sample hospital within each rank of the provincial administrative regions according to the performance appraisal of tertiary public hospitals in 2019 (excellent, good and general). The three representative hospitals were located in each of the provinces.
Participants included all hospital employees on duty during the investigation period. Using an electronic questionnaire, participants directly submitted their responses to a cloud server. Strict confidentiality measures were implemented to ensure the authenticity of this survey data and maintain in a high response rate.
In total, we received 13 211 questionnaires from nine tertiary public hospitals across three provinces in China, accounting for about 35.45% of the total number of employees. Considering the essential roles of doctors and nurses in healthcare delivery, this study focused on the changes in job satisfaction among doctors and nurses. Additionally, results for healthcare professionals across all positions were also reported in the sensitivity analysis to ensure robustness. Among all of the responses, 10 012 (75.79%) were identified as doctors and nurses (further details about the principle of distinguishing were provided in online supplemental file. The proportion of doctors and nurses in 13 211 responses closely mirrors 72.71%, the ratio of doctors and nurses to total hospital staff in tertiary hospitals according to the 2021 China Health Statistical Yearbook, supporting the representativeness of the sample.
Supplemental material
Subsequently, 8417 responses without missing values from 10 012 doctors and nurses (representing 84.07% of 10 012) were included in statistical analyses. Participants were categorised into two groups based on their responses regarding the improvement of the working environment: the exposure group comprised individuals who perceived themselves to be in a ‘more effective’ PATPH working environment, while the control group encompassed the remaining participants. Employing the IPTW approach to mitigate confounding effects arising from hospital and personal characteristics, this study estimated the average treated effect of exposure (ie, the ‘more effective’ PATPH working environment) on job satisfaction. The relationship among all the variables studied was adequately illustrated graphically in online supplemental file.
Patient and public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Measurements
Outcome variable: job satisfaction
The Minnesota Satisfaction Questionnaire (MSQ) (short-form)24 was used to measure healthcare professionals’ job satisfaction. The instrument asked respondents to rate their satisfaction with their present job in 20 aspects which were divided into intrinsic scale (including items 1–4, 7–11, 15–16, 20) and extrinsic scale (including items 5–6, 12–14, 17–19). Responds were made on a 5-point scale from 1 (very dissatisfied) to 5 (very satisfied). The total score with a range of 20–100 was calculated for each participant. The intrinsic and extrinsic subscales were used as outcome measures in the explorational analysis. The Cronbach’s α value of MSQ in this survey was 0.966 (online supplemental table S1).
Exposure variable: the ‘more effective’ PATPH
The determination of a ‘more effective’ PATPH working environment was based on participants’ assessments of the extent to which PATPH improved various aspects of the working environment. Participants rated the improvement across five dimensions of PATPH using a 5-point scale, ranging from 0 (no improvement) to 4 (significant improvement). Responses indicating ‘unclear’ in any dimension were treated as missing values, constituting less than 7.07% in each dimension. For each participant, a total score was calculated by summing their ratings across all five dimensions (ie, medical quality of hospital, operation efficiency of hospital, sustainable development of hospital, satisfaction of inpatients and satisfaction of outpatients). A total score exceeding 18 points (the 75th quantile) was deemed indicative of a ‘more effective’ environment. Participants falling into this category were assigned to the exposed group (assigned a value 1), while those who did not meet this criterion were assigned to the control group (assigned a value of 0). The Cronbach’s α value was 0.970 (online supplemental table S2).
Covariates
We selected covariates based on a priori knowledge of potential factors influencing job satisfaction. These covariates included age group, gender (male or female), marital status (never married or other conditions), position (doctor or nurse), education level (below undergraduate, undergraduate, master’s degree or doctoral degree), technical title (not have, primary title, intermediate title, vice senior or senior), administrative position (have or not have), department (internal medicine, surgical or other departments), region (east, centre or west), performance rating of the hospital (fair, good or excellent), increased attention to the working environment (more or less), depression status of participants (at risk or none). All this information was derived from responses to survey questions and was considered as potential covariates in the analysis.
The increased attention to the working environment among participants was measured by five aspects of PATPH’s indicators (ie, medical quality of hospital, operation efficiency of hospital, sustainable development of hospital, satisfaction of inpatients and satisfaction of outpatients). Responses were elicited on a 5-point scale, ranging from 0 (no increment) to 4 (significant increment). Participants who reported ‘never concern’ in any dimension were classified as missing respondents, constituting less than 6.05% in each dimension. A higher total score indicated a great level of increased attention among participants toward the working environment. Scores less than 9 (representing the 25th quantile) were considered indicative of less increased attention. The Cronbach’s α value was 0.927 (online supplemental table S3).
The depression status of participants as one of the covariates, was measured using the Center for Epidemiological Studies-Depression Scale (CES-D) consisting of 20 items.25 Each item has a 4-point response scale, ranging from 0 (indicating rarely or none of the time) to 3 (indicating all of the time). Four items (4, 8, 12, 16) were reversely scored. Total scores ranged from 0 to 60, with higher scores indicating greater presence of depressive symptoms. Individuals at risk for depression were identified using a cut-off score of 16 or higher.26 The Cronbach’s α value of the CES-D was 0.926.
The levels of anxiety among healthcare professionals as one of the covariates, were assessed using the Self-Rating Anxiety Scale (SAS),27 comprising 20 items. Participants reported the frequency of anxiety-related feelings or behaviours experienced during the past week, with responses recorded on a 4-point scale ranging from 1 (none or almost none) to 4 (almost all the time). Five items (5, 9, 13, 17 and 19) were reverse-scored. The raw scores for all items were summed to calculate a total raw score, which was then multiplied by 1.25 to obtain the standard score. A higher standard score, ranging from 25 to 100, indicated a greater likelihood of experiencing anxiety. Individuals at risk for anxiety disorder were identified using a cut-off score of 50 or higher.28 The Cronbach’s α coefficient of the SAS was 0.877.
Online supplemental figure S1 depicted the causal relationship between the exposure variable and the outcome variable, while also displaying all potential confounding variables measured in this study.
Data analysis
The inverse probability-of-treatment weighting (IPTW) method was applied to mitigate potential confounding effects arising from differences in baseline characteristics between participants exposed to the ‘more effective’ PATPH environment and those who were not (differences in baseline were displayed in figure 1). This method leveraged propensity scores to generate a weight for each participant, assigned weights to individuals based on their propensity scores and created a pseudo-population in which there was no association between baseline observed covariates and the treatment. Subsequently, weighted linear regression analyses were used to estimate the average treatment effect of the ‘more effective’ PATPH working environment on job satisfaction in the pseudo-population.
Standardised differences in proportion between population working in and not in a ‘more effective’ PATPH environment for each baseline covariate before and after IPTW. The solid lines indicate the 10% differences which reflect good balance of confounders; each layer of a binary variable had a standardised difference in proportion with equal value but opposite directions, so only one of them was shown in the figure. Performance represents the performance rating of hospitals; Marriage signifies other marital status other than never married; Educat-Below udg refers education level—below undergraduate; Educat-Udg indicates education level—undergraduate; Title denotes technical title; Depart signifies department; Admin represents administrative position. IPTW, inverse probability-of-treatment weighting; PATPH, Performance Appraisal for Tertiary Public Hospitals.
In detail, this study encompassed three steps: (1) calculate the IPTW weight of each sample based on the propensity score of each sample. (2) Examine the balance of baseline variables before and after applying the IPTW weights through standardised differences. (3) Employ weighted linear regressions to estimate the outcome to mitigate the influence of confounding variables on the results. Figure 2 displays the entire procedure of weight construction, balance diagnosis and the estimation.
Schematic presentation of the overall steps followed in the analysis. PATPH, Performance Appraisal for Tertiary Public Hospitals.
Inverse probability-of-treatment weighting
To enhance statistical efficiency and improve the coverage of CIs, stabilised weights were calculated using the formula29:
where 
represents the stabilised weight for participant i. T denotes the working environment, with t=1 for ‘more effective’ and t=0 for ‘less effective’, i represents participants. C indicates a set of potential confounders. The numerator represents the crude probability of exposure, that is, the probability of being exposed to the ‘more effective’ working environment. The denominator represents the probability of exposure conditioned on the set of potential confounders (ie, the set C).
We chose the traditional strategy of controlled trial-and-error re-specification of the weight-estimating equation in the determination of set C. The exchangeability assumption requires enough joint predictors of exposure and outcome (ie, confounders) in the estimation of the dominator. However, the addition of non-confounding variables may introduce selection bias due to collider stratification, potentially violating the possibility assumption and diminishing statistical efficiency. To achieve a better balance between these considerations, we conducted a backward selection process to include potential confounders.
The optimal set of potential confounders C should contain fewer covariates, resulting in weights with a distribution characterised by a mean close to 1 and a narrower range. These criteria would facilitate better balance across all covariates.30 The primitive set C in the specification one would contain all covariates which showed imbalance (standardised difference in proportion >0.1) at the baseline and affect the probability of T under the consideration of domain knowledge. The standardised differences in proportion were calculated as follow31:
where 
and 
denote the sample prevalence of the exposure (T) in exposed and control groups, respectively.
Extreme weights would be addressed through truncation at the 1st and 99th percentiles in the process of constructing.32 Remaining imbalance after weighing will be addressed in further regression adjustment.33 34
Weighted linear regressions
In weighted linear regressions, the ‘more effective’ PATPH working environment was treated as the treatment variable, while job satisfaction served as the outcome variable. To account for the lack of independence among participants due to IPTW, a robust ‘sandwich’ variance estimator was employed.35
In the primary analysis (model 1), we included all variables that exhibited uneven distributions after IPTW. Alternatively, in model 2, we compared the IPTW approach against a standard stepwise multivariate regression analysis. This comparison allows us to evaluate the performance and effectiveness of the IPTW method in adjusting for confounding compared with traditional regression modelling techniques. Additionally, subgroup analyses were conducted in model 3 and model 4 to investigate potential variations in treatment effects across different regions and levels of increased attention towards the working environment, respectively. Bonferroni corrections were applied in multiple comparisons.
Sensitivity analysis
A series of sensitivity analyses were conducted to test the robustness of the findings. First, we defined ‘more effective’ with different thresholds in models 5–6 (ie, the 50th percentile and the mean+SD). Second, linear mixed-effects models were applied to the weighted population to estimate the impact of the ‘more effective’ working environment, with the region treated as a random effect (model 7). Finally, the sample size was expanded to include medical personnel of all positions in model 8.
R V.4.2.1 was used for data analysis. R package ‘cobalt’ was used to assess the covariate balance. R package ‘MASS’36 and ‘lme4’37 were used to construct the stepwise regression models and the linear mixed-effects model, respectively. R package ‘sandwich’ was used for robust estimation of SE.38 All tests were two-sided with type I error rates of 0.05.
Descriptive analysis and difference significance test
Standard descriptive statistics were used to characterise participants who worked in a ‘more effective’ environment and those who did not. For categorical variables, frequencies and percentages were reported, while continuous variables were summarised using means and SD. To compare differences between groups, the χ2 test was used for categorical variables, examining whether there were significant associations between groups and each categorical variable. For continuous variables, one-way analysis of variance was conducted to assess the significance of differences in means between groups.
Results
Characteristics of participants at baseline
The average age of 8417 participants was 34.02±8.30 years, and males made up 18.64%. Most of the participants had been married previously and doctors constituted 28.15%. According to the definition of ‘more effective’, 2224 (26.42%) participants reported a ‘more effective’ PATPH working environment with a score greater than 18 for working environment improvement. Differences in the distribution of covariates existed at baseline between the ‘more effective’ and ‘less effective’ PATPH working environments (figure 1). Online supplemental table S4 summarised the sociodemographic characteristics of participants. The characteristics of the ‘more effective’ and the ‘less effective’ group of people were also displayed separately in online supplemental table S4.
IPTW weights and balance diagnosis
After several attempts at specifications, this study obtained the optimal stabilised weights using ‘Specification 3_99trunc’ (see online supplemental table S5 for detailed process and see figure 2 for balance diagnoses throughout the entire construction). The optimal set of covariates, denoted as the optimal set C, included age groups, gender, education level, marital status, technical title, position, depression status of healthcare professionals and hospital performance ratings. The mean stabilised weight was 0.99, with an SD of 0.31. The minimum and maximum weights were 0.53 and 2.70, respectively.
The IPTW performed effectively in balancing the baseline covariates. Following IPTW adjustment, the groups increased comparability across most baseline covariates, with standardised differences in proportion being less than 10%. However, there remained residual imbalance in the covariates related to the increased attention to the working environment and the region (figure 1). We addressed the impact of this remaining imbalance by incorporating the two variables into the outcome models (as described later).
Impact of PATPH on job satisfaction
Table 1 shows the impact of the ‘more effective’ PATPH working environment on job satisfaction, demonstrating a nearly 10-point increase in MSQ score (9.57, 95% CI 8.99 to 10.16) in the primary analysis (model 1). Results obtained from the multivariate regression analysis closely aligned with that derived from the IPTW approach (9.92, 95% CI 9.42 to 10.42).
The impact of a ‘more effective’ PATPH working environment on job satisfaction of healthcare professionals using a series of linear regressions
The positive effect of the ‘more effective’ PATPH working environment on job satisfaction across regions exhibited a V-shaped pattern in model 3. Among all regions, the impact of the ‘more effective’ PATPH working environment on MSQ score was most pronounced in the central region, with an increase of approximately 2.04 points compared with the eastern region and about 3.50 points compared with the western region.
Among populations with varying levels of increased attention to the working environment, the average difference in job satisfaction decreased from the group with higher levels of attention to that of those with lower levels. Specifically, the average difference in job satisfaction was 9.60 (95% CI 9.01 to 10.19) for individuals with higher levels of increased attention to the working environment, compared with 8.27 (95% CI 2.57 to 13.96) for individuals with lower levels of increased attention to the working environment.
Sensitivity analysis
The results in table 2 were broadly consistent with those reported in table 1, indicating the robust impact of the ‘more effective’ PATPH on the job satisfaction of healthcare professionals. Balance diagnoses for baseline variables before and after IPTW in the sensitivity analyses were presented in online supplemental figure S3.
Sensitivity analyses of the impact of a ‘more effective’ PATPH working environment on job satisfaction of healthcare professionals
Discussion
‘More effective’ PATPH induced higher job satisfaction
Using IPTW to reduce selective bias, our findings offered a clue that macro health policies such as PATPH played positive roles in enhancing the job satisfaction of healthcare professionals. Doctors and nurses working in environments where PATPH were more effective, experienced an average advancement of approximately 10 points in job satisfaction (equivalent to 12.5% of the range of MSQ score).
This encouraging finding prompted us to investigate which aspects of job satisfaction were predominantly affected. Intrinsic job satisfaction indicated the contentment with the nature of one’s work, while extrinsic job satisfaction encompassed factors such as salary, coworkers and management.39 Conducting exploratory analyses based on model 2, we found that the ‘more effective’ PATPH working environment increased intrinsic MSQ score by approximately 5.46 points (equivalent to 9.10% of the range) and extrinsic MSQ score by approximately 4.46 points (equivalent to 11.15% of the range). The higher proportion of increase in extrinsic MSQ scores in our exploratory results supported our conclusions, implying that a ‘more effective’ PATPH working environment led to greater job satisfaction among doctors and nurses.
PATPH aims to incentivise hospitals to enhance physical and social working conditions, to provide higher medical quality and to improve management. Our findings demonstrated that these anticipated positive impacts were realised to some extent. Plausible explanations for our findings included implemented policies that foster a flexible practice environment with adequate staffing and resources, increased opportunities for healthcare professionals to participate in hospital policies and governance, and more recognition of healthcare professionals’ contributions to work and performance.40 41
Hospitals in the western region exhibit a higher demand for support in improving job satisfaction
Through subgroup analyses, this study evaluated the specific impact among regions characterised by varying economic conditions. When comparing the increase in job satisfaction motivated by PATPH, results in the central and eastern regions showed notable improvements. Nevertheless, in western hospitals, which generally have lower levels of economic development, the impact of environmental improvements on job satisfaction seemed relatively limited. Our finding underscored the necessity for greater support or benefits from PATPH to increase the job satisfaction of healthcare professionals in western regions, in addition to improvements in working settings.
Negative attitude of healthcare professionals diminish the effect of the ‘more effective’ PATPH on job satisfaction
The responses of affected individuals exert considerable influence on the implementation process of policies.42 Model 4 showed our interest in the response of healthcare professionals to PATPH through subgroup analyses. Our findings revealed that when the analysis was restricted to participants with less attention to the working environment, the effect of a ‘more effective’ PATPH declined by approximately 1.30 points in mean MSQ score, accompanied by greater variation. Conversely, the results in the other subgroup, which consists of participants with more increased attention to the working environment, remained consistent with the primary analysis.
During the implementation of policies, it is noted that the involvement level of recipients, service organisations and street-level bureaucrats may influence confidence in and support of policy decisions, thereby enhancing the chances for successful implementation.43 44 We interpreted the less attention to the working environment as a form of negative response from healthcare professionals, indicating a lower engagement level. From an implementation science perspective, our findings highlighted the importance of improving healthcare professionals’ responses to effectively develop, implement and evaluate large-scale healthcare policies. This includes fostering attention, understanding, recognition and support for the policy among healthcare professionals.
Collaborative scheme to enhance motivation among healthcare professionals
Motivating healthcare professionals is a multifaceted endeavour, with various factors influencing their satisfaction.12 45 46 In China, several reforms have been implemented at the individual level to boost the enthusiasm of healthcare professionals, such as reforms in the performance management, promotion system, compensation system. However, despite these efforts, recent studies have shown only marginal improvements in job satisfaction among healthcare professionals in tertiary hospitals in China.47 48 Our study shed light on the potential for the improvements in staff satisfaction pursued by PATPH in hospitals settings, calling for a collaborative motivational scheme that considers both individual-level and environment-level factors.
Enhancing operational efficiency and promoting sustainable development in hospitals are critical elements for maximising the positive effect of an improved environment on job satisfaction. Comparing to higher scores of medical quality and patient satisfaction in the healthcare professionals’ evaluation of PATPH-induced improvements, the operational efficiency and the sustainable development of hospitals scored lower. This is consistent with the historical focus on the provision of medical and public health services in Chinese public hospitals, with fewer efforts directed towards improvements in managerial areas.15 Our finding emphasised the importance of intensifying efforts towards improving operational efficiency and promoting sustainable development in tertiary public hospitals in the future. To achieve this, collaborative efforts are needed to optimise patient flow and enhance coordination among human and material resources. Additionally, ensuring distributive justice to enhance the retention of healthcare personnel is also crucial.49 50
Limitation
Despite the scope and findings of this study, several limitations should be acknowledged. First, our analyses did not include medical personnel in all positions, potentially limiting the generalisability of our findings to broader population of healthcare professionals. Second, the original population of this survey was not equally distributed by covariates. However, we employed IPTW method to diminish the effect of confounders successfully; Third, despite employing IPTW and matching techniques, residual confounding factors may still exist, such as individual preferences of healthcare professionals, income disparities among healthcare professionals, regional customs and cultures. Fourth, our study primarily focused on confirming the overall positive effects of PATPH on job satisfaction. Further research is needed to find which aspects of working environment improved by PATPH contributed to increased job satisfaction, and which aspects of job satisfaction were more affected.
Conclusion
Our study demonstrated that the PATPH, while targeting hospital implementation, had a broader positive impact not only on the working environments of tertiary public hospitals but also on healthcare professionals themselves. We observed that a working environment with more proactive responses to macro policies can lead to a greater increase in job satisfaction. Besides, less individual addition to working environments is linked to lower enhancements in job satisfaction. Findings in this research indicated the impact of macrolevel policies on individual job satisfaction, as well as the moderating effect of individual attention on this impact. We identified an influence pathway beyond policy design, whereby policies promoting hospital improvements subsequently increase staff motivation. Thus, our study advocated for the implementation of a collaborative motivating scheme which takes both individual-level and environment-level factors into account. Such an approach is essential for maximising the positive impact of macrolevel policies on healthcare professionals’ job satisfaction and overall well-being.
This post was originally published on https://bmjopen.bmj.com