Culture happiness (management of cultures) + class happiness (some buildings + civilization value of your city/settlement) + Religion happiness + certain buildings give direct loyalty bonuses (and help with culture/religion assimilation).

Capital province cannot lose loyalty. Province loyalty primarly depends on culture happiness which you can influence by giving rights to a culture. Set a governor of same religion/culture to govern the provinces, increase province happiness by importing goods that effect happiness, change policy of province. If you are a kingdom you can spend PI to boost loyalty by moving an army controlled by your ruler to the province and using the army ability. Buildings for loyalty suck like court house, its better to build building that increase class happiness. Final solution if no other option is available but you have to realize this on time or even in advance make the provinces in question/cultures a vassal state.

If you have cash and plenty of your main pops you can move your culture slaves into newly conquered lands to boost the numbers of your pops in conquered regions - can be expensive, but worth it . Also move the slaves of conquered culture to your main provinces and spread them out across your territorries making sure your culture and religion are dominant in each one to increase the numbers you are converting at once

If I may suggest that when conquering territory from a republic as a kingdom citizens and possibly freeman get a happiness debuff and as republic conquering kingdom territories nobles get a happiness debuff, would nice to add at some point when balacing revolts. Just an idea