Probing anomalous top-Higgs couplings at the HL-LHC viaH→WW^*decay channels

Processes mediated by Flavor Changing Neutral Currents (FCNCs) are very rare in the Standard Model (SM) due to the Glashow-Iliopoulos-Maiani (GIM) mechanism [1]. However, because of the extended flavor structures existing in many New Physics (NP) models, the two-body FCNC decayst $ \to $ qX(q=u/candX=g/γZ/H) can be greatly enhanced: for example, in the Minimal Supersymmetric Standard Model (MSSM) with branching ratioBr(t $ \to $ cH) ~ 10^-5[2], in R-parity violating Supersymmetry (SUSY) with branching ratioBr(t $ \to $ cH) ~ 10^-6[3], in 2-Higgs-Doublet Models (2HDMs) with branching ratioBr(t $ \to $ cH) ~ 10^-5-10^-3[4], in the little Higgs model with T-parity and the warped extra dimensions both with branching ratioBr(t $ \to $ cH) ~ 10^-5[5,6] and so on. Thus any experimental signatures of such FCNC processes will serve as a clear signal for NP Beyond the SM (BSM) [7]. Up to now, top-Higgs FCNC interactions have been studied widely via anomalous top decays or anomalous production processes of single top quark [8-16].

Currently, the ATLAS and CMS collaborations have carried out searches [17-21] fortqHinteractions with 7, 8 and 13 TeV data from the LHC. For example, using 13 TeV data, the ATLAS and the CMS experiments have studied thetqHFCNC processes in top quark pair events withH $ \to $ γγfor ATLAS and $H \to b\bar b$ for CMS. The resulting observed (expected) limits forBr(t $ \to $ qH) at 95% Confidence Level (CL) have been found to be [19,20]:

Very recently, a search for production of top pairs in which one top quark decays viat $ \to $ qHis reported by the ATLAS Collaboration [21], with the subsequent Higgs boson decay to final states with at least one electron or muon. The upper limits on the branching fractionsBr(t $ \to $ Hc) < 0.16% andBr(t $ \to $ Hu) < 0.19% at 95% CL are obtained (with expected limits of 0.15% in both cases). Apart from direct collider measurements, the upper limits ofBr(t $ \to $ qH) < 5 × 10^-3andBr(t $ \to $ qH) < 0.21% can be obtained by bounding thetqHvertex from the observed ${D^0} - {{\bar D}^0}$ mixing [22] and $Z\to c\bar{c}$ [23], respectively.

The upcoming project of the HL-LHC is expected to reach 3 ab^-1. Preliminary sensitivity studies for the HL-LHC suggest the upper bound onBr(t $ \to $ qH) to become about 1.5 × 10^-4at 95% CL by the ATLAS Collaboration [24]. Further, many phenomenological studies within model-independent methods have been performed from different channels [25-33]. In this work, we study the prospects of probing the anomaloustHqcouplings by considering the processes oftHassociated production and $t\bar{t}$ production at the HL-LHC. We analyze two kinds of final states through leptonic top quark decays andH $ \to $ WW^*, one with Same Sign 2-Lepton (SS2L) and the other with 3-Lepton (3L) topology, where the Higgs boson decays into a semi-leptonic (H $ \to $ WW^* $ \to $ $\ell$ ⁺νjj) or fully leptonic ( $H\to W{W}^{\ast }\to {\ell }^{+}\nu {\ell }^{-}\bar{\nu }$ ) mode. The advantage of these channels is that their final states including the SS2L or 3L topologies can be used to significantly suppress QCD backgrounds [34], which have not been fully studied in previous literature.

The organization of this paper is as follows. In Sec. Ⅱ, we discuss two kinds of final states for the processes oftHassociated production with the decay chaint $ \to $ W⁺b $ \to $ $\ell$ ⁺νbandH $ \to $ WW^*as well as $t\bar{t}$ production with the decay chaint $ \to $ $\ell$ ⁺ν _{$\ell$ b}and $\bar{t}\to H(\to W{W}^{\ast })\bar{q}$ . Then we discuss the HL-LHC sensitivity to the anomaloustHqcouplings. We summarize in Sec. Ⅲ.

The general Lagrangian for FCNC top interactions with the Higgs boson can be written as

where the FCNC coupling parameters,κ_tuHandκ_tcH, are real and symmetric since we do not consider here the CP violating effects.

We perform systematic Monte Carlo (MC) simulations and study the sensitivity to the anomaloustHqcouplings through the associatedtHand $t\bar{t}\to tH\bar{q}$ processes at HL-LHC. We first extract the relevant Feynman rules via the FeynRules package [35] and generate the events with MadGraph5-aMC@NLO [36]. The signal and backgrounds samples are simulated at parton level with the NN23LO1 Parton Distribution Function (PDF) set [37] and then passed through PYTHIA6.4 [38] and DELPHES 3 [39] for parton shower and detector simulations, with the MLM matching scheme [40] adopted. Finally, event analysis is performed by using MadAnalysis5 [41].

For the final states including the SS2L topology, the signals are generated through the following processes,

where $\ell$ =e,µ. The representative Feynman diagrams are shown inFig. 1.

Figure 1.Representative Feynman diagrams for the associatedtHprocess (left) and the FCNC decay of the top pair production process (right). Hereq=u,c.

For this channel, the typical signal is exactly two same-sign leptons plus at least three jets, with at least one jet identified asb-jet, and missing transverse energy. The main backgrounds are $t\bar{t}V$ (V=W,Z),W⁺W⁺jjandW⁺Zjj. The $t\bar{t}$ process, which has large cross section, may also contribute to background if a same-sign lepton pair comes from aB-hadron semi-leptonic decay in theb-jet. We do not consider other backgrounds from $t\bar{t}H$ , $t\bar{t}t\bar{t}$ , tri-boson events andtHj. They are neglected because the cross sections are all negligible after applying the selection cuts.

The cross sections of dominant backgrounds at Leading Order (LO) are adjusted to Next-to-LO (NLO) by means ofK-factors, which are 1.04 forW⁺W⁺jjjets [42], 1.24 for $t\bar{t}W$ [43] and 1.39 for [44]. The dominant background is normalized to the NNLO QCD cross section of 953.6 pb [45]. For thetHproduction cross section, the K-factor is taken as 1.5 at the 14 TeV LHC [12].

The decay chainH $ \to $ WW^* $ \to $ $\ell$ νjjmay result in soft leptons and light jets, especially when they are coming from an off-shellWboson. To analyze the signal sensitivity, we thus employ the following basic cuts to select the events:

Basic cuts:p_T( $\ell$ ) > 10 GeV,p_T(j,b) > 15 GeV, |η _{$\ell$ ,j,b}| < 2.5, where $\ell$ =e,µ.

In order to choose appropriate kinematic cuts, we plot inFig. 2examples of kinematic distributions for the signal and backgrounds. Based on these distributions, we impose a further set of cuts.

Figure 2.(color online) Normalized distributions for the signals and the backgrounds.

1) Cut-1: Exactly two same-sign leptons (N( $\ell$ ⁺) = 2) withp_T( $\ell$ ₁) > 20 GeV andp_T( $\ell$ ₂) > 10 GeV ( $\ell$ ₁and $\ell$ ₂denote the higher and lowerp_Tlepton, respectively) plus exactly oneb-tagged jet (N(b) = 1). To remove contamination from hadron decay chains including $\ell$ ⁺ $\ell$ ^-andZboson, we choose the invariant mass larger than 12 GeV and |M _{$\ell$ $\ell$} -m_Z| > 10 GeV.

2) Cut-2: At least two jets in the events are required to be successfully reconstructed, i.e.,N(j)≥2. Among those reconstructed jets, there is at least one pair of jets which could come from aWboson either on-shell or off-shell. Thus the invariant mass of theWboson is required to beM_jj< 90 GeV.

3) Cut-3: The invariant mass ofM _{$\ell$ 2jj}is required to be smaller than 120 GeV.

4) Cut-4: Since the first lepton, $\ell$ ₁, is assumed to originate from the leptonically decaying top quark, the invariant mass of theb-jet and the leading lepton should beM_{b $\ell$ 1}<140 GeV.

5) Cut-5: The scalar sum of transverse momenta,H_T, is to be smaller than 250 GeV.

The effects of the cuts on the signal and background processes are illustrated inTable 1for the SS2L channel, where the anomalous coupling parameters are taken asκ_tuH= 0.1 orκ_tcH= 0.1, while fixing the other to zero. FromTable 1we can see that, after all these cuts, the $t\bar{t}$ backgrounds for the SS2L channel with fake leptons from heavy-flavor jets or charge mis-identifications can be significant.

Obviously, the non-prompt backgrounds may also be significant, where non-prompt leptons are from heavy-flavor decays, mis-identified hadrons, muons from light-meson decays or electrons from un-identified conversions of photons into jets. Recently, the CMS collaboration searched for SS2L signatures [46] and found that the overall non-prompt backgrounds are about 1.5 times the $t\bar{t}W$ background after all cuts. These non-prompt backgrounds are not properly modeled in our MC simulations. Therefore, for simplicity, we add a non-prompt background that is 1.5 times $t\bar{t}W$ [46] after selection cuts to the overall background. Accounting for the theoretical and experimental systematic uncertainties on the background predictions would certainly improve the reliability of the results, yet they can only be neglected in our simulation.

Next, we consider the final states including 3L via the following processes:

where $\ell$ =e,µ.

The dominant SM backgrounds are $t\bar{t}V$ (V=W,Z), $t\bar{t}H$ ,WZ+ jets and $t\bar{t}$ . The multi-jet backgrounds (where jets can fake electrons) are not included since they are negligible in multi-lepton analyses [47].

The pre-selection cuts are taken as follows: there must exist exactly three isolated leptons ( $\ell$ =e,µ) and exactly oneb-tagged jet withp_T( $\ell$ ₁) > 20 GeV,p_T( $\ell$ _{2, 3}) > 10 GeV,p_T(j,b) > 20 GeV, ${\rlap{/}{E}}_{{\rm{T}}}\gt 100$ GeV and |η _$\ell$ ,j,b| < 2.5. These cuts can strongly reduce the $t\bar{t}$ background and its di-boson components.

InFig. 3, we show the invariant mass distribution ofM _{$\ell$ 2 $\ell$ 3}andM_{b $\ell$ 1}from the signal and backgrounds at the 14 TeV LHC. To remove contamination from hadron decay chains including $\ell$ ⁺ $\ell$ ^-pairs and resonantZbosons, we choose the invariant massM _{$\ell$ 2 $\ell$ 3}cuts

Figure 3.(color online) Normalized invariant mass distributions ofM _{$\ell$ 2 $\ell$ 3}(left) andM_{b $\ell$ 1}(right).

Similarly, the invariant mass of theb-jet and the leading lepton,M_{b $\ell$ 1}, should be smaller than 140 GeV. The effects of the cuts on the signal and background processes are illustrated inTable 2for the 3L channel. One can see that significant backgrounds also come from the top pair production process with fake leptons or charge mis-identifications.

Using the Poisson formula [48],

we estimate the Signal Significance (SS) with fixed coupling parametersκ_tqHand a given integrated luminosity ${ {\mathcal L} }_{{\rm{int}}}$ . InFigs. 4and5, we plot the contours ofSS= 3 andSS= 5, respectively, for two channels in the plane of ${ {\mathcal L} }_{{\rm{int}}}-{\kappa }_{tqH}$ . It is clear that, for an integrated luminosity of 3000 fb^-1, the FCNC couplingsκ_tuH(κ_tcH) can be probed to 0.045 (0.052) and 0.035 (0.049) at 3σstatistical sensitivity for the SS2L and 3L channels, respectively. After neglecting the masses of light quarks, the branching ratio oft $ \to $ qHis approximately given by [13,49]

Figure 4.(color online) The 3σcontour plots for the signal in the ${ {\mathcal L} }_{{\rm{int}}}-{\kappa }_{tqH}$ plane for the SS2L (left) and 3L (right) channels at the 14 TeV LHC.

Figure 5.(color online) The 5σcontour plots for the signal in the ${ {\mathcal L} }_{{\rm{int}}}-{\kappa }_{tqH}$ plane for the SS2L (left) and 3L (right) channels at the 14 TeV LHC.

in terms of the Fermi constantG_Fand withx_i=m_i/m_t(i=W,h). In our numerical calculation, the relevant SM input parameters are taken as [50]:

Using Eq. (7), the limits can be translated in terms of constraints on the branching fractions of rare top decays. The 3σCL upper limits onBr(t $ \to $ qH) areBr(t $ \to $ uH) = 1.17 × 10^-3andBr(t $ \to $ cH) = 1.56 × 10^-3for the SS2L channel, andBr(t $ \to $ uH) = 7.1 × 10^-4andBr(t $ \to $ cH) = 1.39 × 10^-3for the 3L channel. The projected limits from different channels are summarized inTable 3. We can see from the table that our results are comparable with the sensitivity limits at the HL-LHC ofBr(t $ \to $ uH) < 0.036% via theH $ \to $ γγchannel [29],Br(t $ \to $ uH) < 0.05% via the multi-lepton channel andBr(t $ \to $ uH) < 0.02% via the di-photon channel [51].

The discovery of the 125 GeV Higgs boson opens the door to probe NP processes that involve Higgs boson associated production or decay. In this paper, we have investigated the signal oftHassociated production via FCNCtqHcouplings and $t\bar{t}$ production with $\bar{t}\to H\bar{q}$ at the 14 TeV LHC. We focused on the final states including SS2L and 3L signals from the decay modest $ \to $ b $\ell$ ⁺ν _$\ell$ ,H $ \to $ WW^* $ \to $ $\ell$ ⁺νjjor $H\to W{W}^{\ast }\to {\ell }^{+}\nu {\ell }^{-}\bar{\nu }$ . We have then shown that, at 3σlevel, the branching ratiosBr(t $ \to $ uH) andBr(t $ \to $ cH) are, respectively, aboutBr(t $ \to $ uH) ≤ 1.17 × 10^-3andBr(t $ \to $ cH) ≤ 1.56 × 10^-3for the SS2L channel, andBr(t $ \to $ uH) ≤ 7.1 × 10^-4andBr(t $ \to $ cH) ≤ 1.39 × 10^-3for the 3L channel at the future HL-LHC.