.. warning:: **Tested against Gammapy version 1.2, please, take into account that might break if Gammapy changes interface** .. _gammapy_plugin: Example to use the Gamma-py plugin with the JeSeT interface =========================================================== In this tutorial we show how to import a jetset model into Gamma-py, and finally we perform a model fitting with Gamma-py. To run this plugin you have to install Gamma-py https://docs.gammapy.org/0.19/getting-started/install.html .. code:: ipython3 import astropy.units as u import numpy as np import matplotlib.pyplot as plt import matplotlib as mpl mpl.rcParams['figure.dpi'] = 80 from jetset.gammapy_plugin import GammapyJetsetModelFactory from jetset.jet_model import Jet from jetset.test_data_helper import test_SEDs from jetset.data_loader import ObsData,Data from jetset.plot_sedfit import PlotSED from jetset.test_data_helper import test_SEDs Importing a jetset model into gammapy ------------------------------------- .. code:: ipython3 jet=Jet() .. parsed-literal:: ===> setting C threads to 12 .. code:: ipython3 jet.parameters .. raw:: html
Table length=11
model namenamepar typeunitsvalphys. bound. minphys. bound. maxlogfrozen
str12str16str21objectfloat64float64float64boolbool
jet_leptonicRregion_sizecm5.000000e+151.000000e+031.000000e+30FalseFalse
jet_leptonicR_Hregion_positioncm1.000000e+170.000000e+00--FalseTrue
jet_leptonicBmagnetic_fieldgauss1.000000e-010.000000e+00--FalseFalse
jet_leptonicNH_cold_to_rel_ecold_p_to_rel_e_ratio1.000000e+000.000000e+00--FalseTrue
jet_leptonicbeam_objbeaming1.000000e+011.000000e-04--FalseFalse
jet_leptonicz_cosmredshift1.000000e-010.000000e+00--FalseFalse
jet_leptonicgminlow-energy-cut-offlorentz-factor*2.000000e+001.000000e+001.000000e+09FalseFalse
jet_leptonicgmaxhigh-energy-cut-offlorentz-factor*1.000000e+061.000000e+001.000000e+15FalseFalse
jet_leptonicNemitters_density1 / cm31.000000e+020.000000e+00--FalseFalse
jet_leptonicgamma_cutturn-over-energylorentz-factor*1.000000e+041.000000e+001.000000e+09FalseFalse
jet_leptonicpLE_spectral_slope2.000000e+00-1.000000e+011.000000e+01FalseFalse
.. parsed-literal:: None .. code:: ipython3 gammapy_jet_model=GammapyJetsetModelFactory(jet) gammapy_jet_model.parameters.to_table() .. parsed-literal:: ===> setting C threads to 12 .. raw:: html
Table length=12
typenamevalueuniterrorminmaxfrozenis_normlinkprior
str1str16float64str4float64float64float64boolboolstr1str1
gmin2.0000e+000.000e+001.000e+001.000e+09FalseFalse
gmax1.0000e+060.000e+001.000e+001.000e+15FalseFalse
N1.0000e+02cm-30.000e+000.000e+00nanFalseFalse
gamma_cut1.0000e+040.000e+001.000e+001.000e+09FalseFalse
p2.0000e+000.000e+00-1.000e+011.000e+01FalseFalse
R5.0000e+15cm0.000e+001.000e+031.000e+30FalseFalse
R_H1.0000e+17cm0.000e+000.000e+00nanTrueFalse
B1.0000e-01G0.000e+000.000e+00nanFalseFalse
NH_cold_to_rel_e1.0000e+000.000e+000.000e+00nanTrueFalse
beam_obj1.0000e+010.000e+001.000e-04nanFalseFalse
z_cosm1.0000e-010.000e+000.000e+00nanFalseFalse
fake_norm1.0000e+000.000e+000.000e+00nanTrueTrue
let’s verify that parameters are updated .. code:: ipython3 gammapy_jet_model.R.value=1E15 gammapy_jet_model.N.value=1E4 gammapy_jet_model.p.value=1.5 .. code:: ipython3 gammapy_jet_model.parameters.to_table() .. raw:: html
Table length=12
typenamevalueuniterrorminmaxfrozenis_normlinkprior
str1str16float64str4float64float64float64boolboolstr1str1
gmin2.0000e+000.000e+001.000e+001.000e+09FalseFalse
gmax1.0000e+060.000e+001.000e+001.000e+15FalseFalse
N1.0000e+04cm-30.000e+000.000e+00nanFalseFalse
gamma_cut1.0000e+040.000e+001.000e+001.000e+09FalseFalse
p1.5000e+000.000e+00-1.000e+011.000e+01FalseFalse
R1.0000e+15cm0.000e+001.000e+031.000e+30FalseFalse
R_H1.0000e+17cm0.000e+000.000e+00nanTrueFalse
B1.0000e-01G0.000e+000.000e+00nanFalseFalse
NH_cold_to_rel_e1.0000e+000.000e+000.000e+00nanTrueFalse
beam_obj1.0000e+010.000e+001.000e-04nanFalseFalse
z_cosm1.0000e-010.000e+000.000e+00nanFalseFalse
fake_norm1.0000e+000.000e+000.000e+00nanTrueTrue
plotting with gammapy ~~~~~~~~~~~~~~~~~~~~~ .. code:: ipython3 p=gammapy_jet_model.plot(energy_bounds=[1E-18, 10] * u.TeV,energy_power=2) .. image:: gammapy_plugin_files/gammapy_plugin_13_0.png plotting with jetset ~~~~~~~~~~~~~~~~~~~~ .. code:: ipython3 gammapy_jet_model.jetset_model.plot_model() .. parsed-literal:: .. image:: gammapy_plugin_files/gammapy_plugin_15_1.png Model fitting with gammapy -------------------------- .. code:: ipython3 %matplotlib inline data=Data.from_file(test_SEDs[1]) sed_data=ObsData(data_table=data) sed_data.group_data(bin_width=0.1) sed_data.add_systematics(0.1,[10.**6,10.**29]) p=sed_data.plot_sed() .. parsed-literal:: ================================================================================ *** binning data *** ---> N bins= 179 ---> bin_widht= 0.1 msk [False True True False False True False True True True True True False True True False False False False False False True False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True False True True False False False False False False False False False False False False False False False False True True True True True True True False False True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True False False False True False False False True False False False True False False False True False False False True False False False True False False False True False True False True False True False True False True False True False True False True False False] ================================================================================ .. image:: gammapy_plugin_files/gammapy_plugin_17_1.png .. code:: ipython3 from jetset.sed_shaper import SEDShape my_shape=SEDShape(sed_data) my_shape.eval_indices(minimizer='lsb',silent=True) p=my_shape.plot_indices() p.setlim(y_min=1E-15,y_max=1E-6) .. parsed-literal:: ================================================================================ *** evaluating spectral indices for data *** ================================================================================ .. image:: gammapy_plugin_files/gammapy_plugin_18_1.png .. code:: ipython3 mm,best_fit=my_shape.sync_fit(check_host_gal_template=False, Ep_start=None, minimizer='lsb', silent=True, fit_range=[10.,21.]) .. parsed-literal:: ================================================================================ *** Log-Polynomial fitting of the synchrotron component *** ---> first blind fit run, fit range: [10.0, 21.0] ---> class: HSP .. raw:: html
Table length=4
model namenamevalbestfit valerr +err -start valfit range minfit range maxfrozen
str8str2float64float64float64float64float64float64float64bool
LogCubicb-1.654034e-01-1.654034e-014.639280e-03---1.000000e+00-1.000000e+010.000000e+00False
LogCubicc-1.194746e-02-1.194746e-026.870736e-04---1.000000e+00-1.000000e+011.000000e+01False
LogCubicEp1.673186e+011.673186e+011.710428e-02--1.668578e+010.000000e+003.000000e+01False
LogCubicSp-9.478048e+00-9.478048e+001.351655e-02---1.000000e+01-3.000000e+010.000000e+00False
.. parsed-literal:: ---> sync nu_p=+1.673186e+01 (err=+1.710428e-02) nuFnu_p=-9.478048e+00 (err=+1.351655e-02) curv.=-1.654034e-01 (err=+4.639280e-03) ================================================================================ .. code:: ipython3 my_shape.IC_fit(fit_range=[23.,29.],minimizer='minuit',silent=True) p=my_shape.plot_shape_fit() p.setlim(y_min=1E-15) .. parsed-literal:: ================================================================================ *** Log-Polynomial fitting of the IC component *** ---> fit range: [23.0, 29.0] ---> LogCubic fit ====> simplex ====> migrad ====> simplex ====> migrad ====> simplex ====> migrad .. raw:: html
Table length=4
model namenamevalbestfit valerr +err -start valfit range minfit range maxfrozen
str8str2float64float64float64float64float64float64float64bool
LogCubicb-2.003642e-01-2.003642e-012.690887e-02---1.000000e+00-1.000000e+010.000000e+00False
LogCubicc-4.156240e-02-4.156240e-022.110793e-02---1.000000e+00-1.000000e+011.000000e+01False
LogCubicEp2.522232e+012.522232e+011.174807e-01--2.529619e+010.000000e+003.000000e+01False
LogCubicSp-1.012086e+01-1.012086e+013.053770e-02---1.000000e+01-3.000000e+010.000000e+00False
.. parsed-literal:: ---> IC nu_p=+2.522232e+01 (err=+1.174807e-01) nuFnu_p=-1.012086e+01 (err=+3.053770e-02) curv.=-2.003642e-01 (err=+2.690887e-02) ================================================================================ .. image:: gammapy_plugin_files/gammapy_plugin_20_3.png .. code:: ipython3 from jetset.obs_constrain import ObsConstrain from jetset.model_manager import FitModel sed_obspar=ObsConstrain(beaming=25, B_range=[0.001,0.1], distr_e='lppl', t_var_sec=3*86400, nu_cut_IR=1E12, SEDShape=my_shape) prefit_jet=sed_obspar.constrain_SSC_model(electron_distribution_log_values=False,silent=True) prefit_jet.save_model('prefit_jet.pkl') .. parsed-literal:: ================================================================================ *** constrains parameters from observable *** ===> setting C threads to 12 .. parsed-literal:: /Users/orion/miniforge3/envs/jetset/lib/python3.10/site-packages/jetset/obs_constrain.py:1150: RankWarning: Polyfit may be poorly conditioned p=polyfit(nu_p_IC_model_log,B_grid_log,2) .. raw:: html
Table length=12
model namenamepar typeunitsvalphys. bound. minphys. bound. maxlogfrozen
str12str16str21objectfloat64float64float64boolbool
jet_leptonicRregion_sizecm3.484042e+161.000000e+031.000000e+30FalseFalse
jet_leptonicR_Hregion_positioncm1.000000e+170.000000e+00--FalseTrue
jet_leptonicBmagnetic_fieldgauss5.050000e-020.000000e+00--FalseFalse
jet_leptonicNH_cold_to_rel_ecold_p_to_rel_e_ratio1.000000e+000.000000e+00--FalseTrue
jet_leptonicbeam_objbeaming2.500000e+011.000000e-04--FalseFalse
jet_leptonicz_cosmredshift3.080000e-020.000000e+00--FalseFalse
jet_leptonicgminlow-energy-cut-offlorentz-factor*4.697542e+021.000000e+001.000000e+09FalseFalse
jet_leptonicgmaxhigh-energy-cut-offlorentz-factor*1.373160e+061.000000e+001.000000e+15FalseFalse
jet_leptonicNemitters_density1 / cm36.040733e-010.000000e+00--FalseFalse
jet_leptonicgamma0_log_parabturn-over-energylorentz-factor*3.404403e+041.000000e+001.000000e+09FalseFalse
jet_leptonicsLE_spectral_slope2.163458e+00-1.000000e+011.000000e+01FalseFalse
jet_leptonicrspectral_curvature8.270168e-01-1.500000e+011.500000e+01FalseFalse
.. parsed-literal:: ================================================================================ .. code:: ipython3 pl=prefit_jet.plot_model(sed_data=sed_data) pl.add_model_residual_plot(prefit_jet,sed_data) pl.setlim(y_min=1E-15,x_min=1E7,x_max=1E29) .. image:: gammapy_plugin_files/gammapy_plugin_22_0.png setting gammapy jetset model ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We import the model to gammapy and we set min/max values. Notice that gammapy has not fit_range, but uses only min/max. We importing a jetset model with ``fit_range`` defined, these will automatically update the gammapy min/max parameters attributes .. code:: ipython3 jet=Jet.load_model('prefit_jet.pkl') jet.parameters.z_cosm.freeze() jet.parameters.R_H.freeze() jet.parameters.R.freeze() jet.parameters.gmin.freeze() #jet.parameters.R.fit_range=[5E15,1E17] #jet.parameters.gmin.fit_range=[10,1000] jet.parameters.gmax.fit_range=[1E5,1E7] jet.parameters.s.fit_range=[1,3] jet.parameters.r.fit_range=[0,5] jet.parameters.B.fit_range=[1E-4,1] jet.parameters.N.fit_range=[1E-3,10] jet.parameters.gamma0_log_parab.fit_range=[1E3,1E5] jet.parameters.beam_obj.fit_range=[5,50] gammapy_jet_model=GammapyJetsetModelFactory(jet) .. parsed-literal:: ===> setting C threads to 12 ===> setting C threads to 12 .. code:: ipython3 gammapy_jet_model.parameters.to_table() .. raw:: html
Table length=13
typenamevalueuniterrorminmaxfrozenis_normlinkprior
str1str16float64str4float64float64float64boolboolstr1str1
gmin4.6975e+020.000e+001.000e+001.000e+09TrueFalse
gmax1.3732e+060.000e+001.000e+051.000e+07FalseFalse
N6.0407e-01cm-30.000e+001.000e-031.000e+01FalseFalse
gamma0_log_parab3.4044e+040.000e+001.000e+031.000e+05FalseFalse
s2.1635e+000.000e+001.000e+003.000e+00FalseFalse
r8.2702e-010.000e+000.000e+005.000e+00FalseFalse
R3.4840e+16cm0.000e+001.000e+031.000e+30TrueFalse
R_H1.0000e+17cm0.000e+000.000e+00nanTrueFalse
B5.0500e-02G0.000e+001.000e-041.000e+00FalseFalse
NH_cold_to_rel_e1.0000e+000.000e+000.000e+00nanTrueFalse
beam_obj2.5000e+010.000e+005.000e+005.000e+01FalseFalse
z_cosm3.0800e-020.000e+000.000e+00nanTrueFalse
fake_norm1.0000e+000.000e+000.000e+00nanTrueTrue
.. code:: ipython3 _=gammapy_jet_model.evaluate() .. code:: ipython3 p=gammapy_jet_model.jetset_model.plot_model(sed_data=sed_data) p.add_model_residual_plot(data=sed_data, model=jet,fit_range=[1E11,1E30]) p.setlim(x_min=1E8,y_min=1E-14) .. image:: gammapy_plugin_files/gammapy_plugin_28_0.png importing data to gammapy ~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: ipython3 from gammapy.estimators import FluxPoints fp=FluxPoints.from_table(sed_data.gammapy_table,sed_type='e2dnde', format='gadf-sed') p=fp.plot(sed_type='e2dnde') p=gammapy_jet_model.plot(energy_bounds=[1E-18, 10] * u.TeV,energy_power=2) plt.show() .. parsed-literal:: No reference model set for FluxMaps. Assuming point source with E^-2 spectrum. .. image:: gammapy_plugin_files/gammapy_plugin_30_1.png .. code:: ipython3 sed_data.gammapy_table.meta .. parsed-literal:: OrderedDict([('z', 0.0308), ('obj_name', 'J1104+3812,Mrk421'), ('restframe', 'obs'), ('data_scale', 'lin-lin'), ('UL_CL', 0.95), ('SED_TYPE', 'e2dnde')]) .. code:: ipython3 p=fp.plot(sed_type='dnde') p=gammapy_jet_model.plot(energy_bounds=[1E-18, 10] * u.TeV,energy_power=0) plt.show() .. image:: gammapy_plugin_files/gammapy_plugin_32_0.png building gammapy SkyModel ~~~~~~~~~~~~~~~~~~~~~~~~~ we build the SkyModel, and we degrade the pre-fit model quality .. code:: ipython3 from gammapy.modeling.models import SkyModel sky_model = SkyModel(name="SSC model Mrk 421", spectral_model=gammapy_jet_model) gammapy_jet_model.N.value=2.0 gammapy_jet_model.r.value=0.5 gammapy_jet_model.beam_obj.value=20 print(sky_model) gammapy_jet_model.evaluate() p=gammapy_jet_model.jetset_model.plot_model(sed_data=sed_data) p.add_model_residual_plot(data=sed_data, model=gammapy_jet_model.jetset_model,fit_range=[1E11,1E30]) .. parsed-literal:: SkyModel Name : SSC model Mrk 421 Datasets names : None Spectral model type : GammapyJetsetModel Spatial model type : Temporal model type : Parameters: gmin (frozen): 469.754 gmax : 1373159.756 +/- 0.00 N : 2.000 +/- 0.00 1 / cm3 gamma0_log_parab : 34044.032 +/- 0.00 s : 2.163 +/- 0.00 r : 0.500 +/- 0.00 R (frozen): 34840420166069272.000 cm R_H (frozen): 100000000000000000.000 cm B : 0.051 +/- 0.00 gauss NH_cold_to_rel_e (frozen): 1.000 beam_obj : 20.000 +/- 0.00 z_cosm (frozen): 0.031 fake_norm (frozen): 1.000 .. image:: gammapy_plugin_files/gammapy_plugin_36_1.png setting gammapy Datasets and Fit classes, and running the fit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: ipython3 from gammapy.datasets import FluxPointsDataset,Datasets datasets = Datasets() E_min_fit = (1e11 * u.Hz).to("eV", equivalencies=u.spectral()) fp=FluxPoints.from_table(sed_data.gammapy_table,sed_type='e2dnde', format='gadf-sed') dataset_mrk421 = FluxPointsDataset(data=fp,models=sky_model) #this workaround was needed with version 1.2 dataset_mrk421.mask_fit= dataset_mrk421.data.energy_ref >= E_min_fit dataset_mrk421.mask_fit=dataset_mrk421.mask_fit.reshape(dataset_mrk421.mask_safe.shape) datasets = Datasets(dataset_mrk421) datasets.models=sky_model .. parsed-literal:: No reference model set for FluxMaps. Assuming point source with E^-2 spectrum. .. code:: ipython3 dataset_mrk421.mask_fit.shape .. parsed-literal:: (56, 1, 1) .. code:: ipython3 from gammapy.modeling import Fit #conf_dict=dict(tol=1E-8) fitter = Fit(backend='scipy')#,optimize_opts=conf_dict) .. code:: ipython3 results = fitter.run(datasets=datasets) print(results) .. parsed-literal:: ===> setting C threads to 12 .. parsed-literal:: No covariance estimate - not supported by this backend. .. parsed-literal:: OptimizeResult backend : scipy method : scipy success : True message : Optimization terminated successfully. nfev : 1307 total stat : 38.09 .. code:: ipython3 results.parameters.to_table() .. raw:: html
Table length=13
typenamevalueuniterrorminmaxfrozenis_normlinkprior
str1str16float64str4float64float64float64boolboolstr1str1
gmin4.6975e+020.000e+001.000e+001.000e+09TrueFalse
gmax9.7813e+050.000e+001.000e+051.000e+07FalseFalse
N5.2594e-01cm-30.000e+001.000e-031.000e+01FalseFalse
gamma0_log_parab3.5520e+040.000e+001.000e+031.000e+05FalseFalse
s2.0751e+000.000e+001.000e+003.000e+00FalseFalse
r7.8628e-010.000e+000.000e+005.000e+00FalseFalse
R3.4840e+16cm0.000e+001.000e+031.000e+30TrueFalse
R_H1.0000e+17cm0.000e+000.000e+00nanTrueFalse
B4.9767e-02G0.000e+001.000e-041.000e+00FalseFalse
NH_cold_to_rel_e1.0000e+000.000e+000.000e+00nanTrueFalse
beam_obj2.2991e+010.000e+005.000e+005.000e+01FalseFalse
z_cosm3.0800e-020.000e+000.000e+00nanTrueFalse
fake_norm1.0000e+000.000e+000.000e+00nanTrueTrue
.. code:: ipython3 gammapy_jet_model.jetset_model.parameters .. raw:: html
Table length=13
model namenamepar typeunitsvalphys. bound. minphys. bound. maxlogfrozen
str12str16str21objectfloat64float64float64boolbool
jet_leptonicgminlow-energy-cut-offlorentz-factor*4.697542e+021.000000e+001.000000e+09FalseTrue
jet_leptonicgmaxhigh-energy-cut-offlorentz-factor*9.781287e+051.000000e+001.000000e+15FalseFalse
jet_leptonicNemitters_density1 / cm35.259358e-010.000000e+00--FalseFalse
jet_leptonicgamma0_log_parabturn-over-energylorentz-factor*3.552033e+041.000000e+001.000000e+09FalseFalse
jet_leptonicsLE_spectral_slope2.075093e+00-1.000000e+011.000000e+01FalseFalse
jet_leptonicrspectral_curvature7.862822e-01-1.500000e+011.500000e+01FalseFalse
jet_leptonicRregion_sizecm3.484042e+161.000000e+031.000000e+30FalseTrue
jet_leptonicR_Hregion_positioncm1.000000e+170.000000e+00--FalseTrue
jet_leptonicBmagnetic_fieldgauss4.976724e-020.000000e+00--FalseFalse
jet_leptonicNH_cold_to_rel_ecold_p_to_rel_e_ratio1.000000e+000.000000e+00--FalseTrue
jet_leptonicbeam_objbeaming2.299085e+011.000000e-04--FalseFalse
jet_leptonicz_cosmredshift3.080000e-020.000000e+00--FalseTrue
jet_leptonicfake_normuser_defined1.000000e+000.000000e+00--FalseTrue
.. parsed-literal:: None note that this plot refers to the latest fit trial, in case, please consider storing the plot within a list in the fit loop .. code:: ipython3 gammapy_jet_model.evaluate() fp.plot(sed_type='e2dnde') gammapy_jet_model.plot(energy_bounds=[1E-18, 10] * u.TeV,energy_power=2) plt.ylim(1E-14) plt.show() .. image:: gammapy_plugin_files/gammapy_plugin_45_0.png .. code:: ipython3 gammapy_jet_model.jetset_model.eval() p=gammapy_jet_model.jetset_model.plot_model(sed_data=sed_data) p.add_model_residual_plot(data=sed_data, model=gammapy_jet_model.jetset_model, fit_range=[1E11,1E30]) p.setlim(y_min=1E-14) .. image:: gammapy_plugin_files/gammapy_plugin_46_0.png .. code:: ipython3 %timeit gammapy_jet_model.jetset_model.eval() .. parsed-literal:: 27.1 ms ± 357 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) .. code:: ipython3 %timeit gammapy_jet_model.evaluate() .. parsed-literal:: 28.8 ms ± 267 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)