Example to use the Sherpa minimizer plugin with a JeSeT model#

In this tutorial we show how to import a jetset model into sherpa, and finally we perform a model fitting. To run this plugin you have to install Sherpa: https://sherpa.readthedocs.io/en/latest/install.html

import jetset
print('tested with',jetset.__version__)

tested with 1.4.0rc0

import warnings
warnings.filterwarnings('ignore')

import matplotlib.pylab as plt
import jetset
from jetset.test_data_helper import  test_SEDs
from jetset.data_loader import ObsData,Data
from jetset.plot_sedfit import PlotSED
from jetset.test_data_helper import  test_SEDs

test_SEDs

['/Users/orion/miniforge3/envs/jetset/lib/python3.12/site-packages/jetset/test_data/SEDs_data/SED_3C345.ecsv',
 '/Users/orion/miniforge3/envs/jetset/lib/python3.12/site-packages/jetset/test_data/SEDs_data/SED_MW_Mrk421_EBL_DEABS.ecsv',
 '/Users/orion/miniforge3/envs/jetset/lib/python3.12/site-packages/jetset/test_data/SEDs_data/SED_MW_Mrk501_EBL_ABS.ecsv',
 '/Users/orion/miniforge3/envs/jetset/lib/python3.12/site-packages/jetset/test_data/SEDs_data/SED_MW_Mrk501_EBL_DEABS.ecsv']

Loading data#

see the Data format and SED data user guide for further information about loading data

print(test_SEDs[2])
data=Data.from_file(test_SEDs[2])

/Users/orion/miniforge3/envs/jetset/lib/python3.12/site-packages/jetset/test_data/SEDs_data/SED_MW_Mrk501_EBL_ABS.ecsv

%matplotlib inline
sed_data=ObsData(data_table=data)
sed_data.group_data(bin_width=0.2)

sed_data.add_systematics(0.1,[10.**6,10.**29])
p=sed_data.plot_sed()

================================================================================

*  binning data  *
---> N bins= 88
---> bin_width= 0.2
================================================================================

../../../../_images/sherpa-plugin-jetset-interface_9_1.png

sed_data.save('Mrk_501.pkl')

phenomenological model constraining#

see the Phenomenological model constraining: application user guide for further information about phenomenological constraining

spectral indices#

from jetset.sed_shaper import  SEDShape
my_shape=SEDShape(sed_data)
my_shape.eval_indices(minimizer='lsb',silent=True)
p=my_shape.plot_indices()
p.setlim(y_min=1E-15,y_max=1E-6)

================================================================================

* evaluating spectral indices for data *
================================================================================

../../../../_images/sherpa-plugin-jetset-interface_14_1.png

sed shaper#

mm,best_fit=my_shape.sync_fit(check_host_gal_template=True,
                  Ep_start=None,
                  minimizer='lsb',
                  silent=True,
                  fit_range=[10.,21.])

================================================================================

* Log-Polynomial fitting of the synchrotron component *
---> first blind fit run,  fit range: [10.0, 21.0]
---> class:  HSP

---> class:  HSP

Table length=6

model name	name	val	bestfit val	err +	err -	start val	fit range min	fit range max	frozen
LogCubic	b	-5.589204e-02	-5.589204e-02	6.234447e-03	--	-5.237376e-02	-1.000000e+01	0.000000e+00	False
LogCubic	c	-3.292704e-04	-3.292704e-04	8.967926e-04	--	7.639964e-03	-1.000000e+01	1.000000e+01	False
LogCubic	Ep	1.698160e+01	1.698160e+01	8.582826e-02	--	1.556163e+01	0.000000e+00	3.000000e+01	False
LogCubic	Sp	-1.030620e+01	-1.030620e+01	1.607947e-02	--	-1.019011e+01	-3.000000e+01	0.000000e+00	False
host_galaxy	nuFnu_p_host	-9.972939e+00	-9.972939e+00	3.177493e-02	--	-1.019011e+01	-1.219011e+01	-8.190114e+00	False
host_galaxy	nu_scale	-2.290459e-03	-2.290459e-03	1.811351e-03	--	0.000000e+00	-2.908697e-03	2.908697e-03	False

---> sync       nu_p=+1.698160e+01 (err=+8.582826e-02)  nuFnu_p=-1.030620e+01 (err=+1.607947e-02) curv.=-5.589204e-02 (err=+6.234447e-03)
================================================================================

my_shape.IC_fit(fit_range=[23.,29.],minimizer='minuit',silent=True)
p=my_shape.plot_shape_fit()
p.setlim(y_min=1E-15)

================================================================================

* Log-Polynomial fitting of the IC component *
---> fit range: [23.0, 29.0]
---> LogCubic fit

Table length=4

model name	name	val	bestfit val	err +	err -	start val	fit range min	fit range max	frozen
LogCubic	b	-1.627265e-01	-1.627265e-01	2.483991e-02	--	-1.000000e+00	-1.000000e+01	0.000000e+00	False
LogCubic	c	-4.569970e-02	-4.569970e-02	1.984551e-02	--	-1.000000e+00	-1.000000e+01	1.000000e+01	False
LogCubic	Ep	2.526444e+01	2.526444e+01	1.736457e-01	--	2.531190e+01	0.000000e+00	3.000000e+01	False
LogCubic	Sp	-1.057790e+01	-1.057790e+01	4.914805e-02	--	-1.000000e+01	-3.000000e+01	0.000000e+00	False

---> IC         nu_p=+2.526444e+01 (err=+1.736457e-01)  nuFnu_p=-1.057790e+01 (err=+4.914805e-02) curv.=-1.627265e-01 (err=+2.483991e-02)
================================================================================

../../../../_images/sherpa-plugin-jetset-interface_17_3.png

Model constraining#

In this step we are not fitting the model, we are just obtaining the phenomenological pre_fit model, that will be fitted in using minuit ore least-square bound, as shown below

from jetset.obs_constrain import ObsConstrain
from jetset.model_manager import  FitModel
sed_obspar=ObsConstrain(beaming=25,
                        B_range=[0.001,0.1],
                        distr_e='lppl',
                        t_var_sec=3*86400,
                        nu_cut_IR=1E12,
                        SEDShape=my_shape)


prefit_jet=sed_obspar.constrain_SSC_model(electron_distribution_log_values=False,silent=True)
prefit_jet.save_model('prefit_jet.pkl')

================================================================================

*  constrains parameters from observable *

/Users/orion/miniforge3/envs/jetset/lib/python3.12/site-packages/jetset/obs_constrain.py:1514: RankWarning: Polyfit may be poorly conditioned
  p=polyfit(nu_p_IC_model_log,B_grid_log,2)

Table length=12

model name	name	par type	units	val	phys. bound. min	phys. bound. max	log	frozen
jet_leptonic	R	region_size	cm	1.216622e+16	1.000000e+03	1.000000e+30	False	False
jet_leptonic	R_H	region_position	cm	1.000000e+17	0.000000e+00	--	False	True
jet_leptonic	B	magnetic_field	gauss	5.050000e-02	0.000000e+00	--	False	False
jet_leptonic	NH_cold_to_rel_e	cold_p_to_rel_e_ratio		1.000000e+00	0.000000e+00	--	False	True
jet_leptonic	beam_obj	beaming		2.500000e+01	1.000000e-04	--	False	False
jet_leptonic	z_cosm	redshift		3.360000e-02	0.000000e+00	--	False	False
jet_leptonic	gmin	low-energy-cut-off	lorentz-factor*	4.703917e+02	1.000000e+00	1.000000e+09	False	False
jet_leptonic	gmax	high-energy-cut-off	lorentz-factor*	2.121873e+06	1.000000e+00	1.000000e+15	False	False
jet_leptonic	N	emitters_density	1 / cm3	5.583988e+00	0.000000e+00	--	False	False
jet_leptonic	gamma0_log_parab	turn-over-energy	lorentz-factor*	6.673207e+03	1.000000e+00	1.000000e+09	False	False
jet_leptonic	s	LE_spectral_slope		2.251647e+00	-1.000000e+01	1.000000e+01	False	False
jet_leptonic	r	spectral_curvature		2.794602e-01	-1.500000e+01	1.500000e+01	False	False

================================================================================

pl=prefit_jet.plot_model(sed_data=sed_data)
pl.add_model_residual_plot(prefit_jet,sed_data)
pl.setlim(y_min=1E-15,x_min=1E7,x_max=1E29)

../../../../_images/sherpa-plugin-jetset-interface_21_0.png

Model fitting with Sherpa#

from jetset.template_2Dmodel import EBLAbsorptionTemplate
ebl_franceschini=EBLAbsorptionTemplate.from_name('Franceschini_2008')

from jetset.jet_model import Jet
prefit_jet=Jet.load_model('prefit_jet.pkl')

composite_model=FitModel(nu_size=500,name='EBL corrected',template=my_shape.host_gal)
composite_model.add_component(prefit_jet)
composite_model.add_component(ebl_franceschini)
composite_model.link_par(par_name='z_cosm', from_model='Franceschini_2008', to_model='jet_leptonic')
composite_model.composite_expr='(jet_leptonic+host_galaxy)*Franceschini_2008'
composite_model.eval()
composite_model.plot_model()

<jetset.plot_sedfit.PlotSED at 0x13583a750>

../../../../_images/sherpa-plugin-jetset-interface_25_1.png

composite_model.freeze('jet_leptonic','z_cosm')
composite_model.freeze('jet_leptonic','R_H')
composite_model.jet_leptonic.parameters.beam_obj.fit_range=[5., 50.]
composite_model.jet_leptonic.parameters.R.fit_range=[10**15.5,10**17.5]
composite_model.jet_leptonic.parameters.gmax.fit_range=[1E5,1E7]
composite_model.host_galaxy.parameters.nuFnu_p_host.frozen=False
composite_model.host_galaxy.parameters.nu_scale.frozen=True

from jetset.minimizer import ModelMinimizer
composite_model.jet_leptonic.parameters.z_cosm.frozen=True

model_minimizer_lsb=ModelMinimizer('sherpa')
best_fit=model_minimizer_lsb.fit(composite_model,sed_data,1E11,1E29,fitname='SSC-best-fit-sherpa',repeat=1)

filtering data in fit range = [1.000000e+11,1.000000e+29]
data length 31
================================================================================

* start fit process *
-----

0it [00:00, ?it/s]

jetset model name R renamed to  R_sh due to sherpa internal naming convention
- best chisq=1.39904e+01

-------------------------------------------------------------------------
Fit report

Model: SSC-best-fit-sherpa

Table length=16

model name	name	par type	units	val	phys. bound. min	phys. bound. max	log	frozen
host_galaxy	nuFnu_p_host	nuFnu-scale	erg / (s cm2)	-9.967750e+00	-2.000000e+01	2.000000e+01	False	False
host_galaxy	nu_scale	nu-scale	Hz	-2.290459e-03	-2.000000e+00	2.000000e+00	False	True
jet_leptonic	gmin	low-energy-cut-off	lorentz-factor*	2.850763e+02	1.000000e+00	1.000000e+09	False	False
jet_leptonic	gmax	high-energy-cut-off	lorentz-factor*	1.938561e+06	1.000000e+00	1.000000e+15	False	False
jet_leptonic	N	emitters_density	1 / cm3	6.150849e+00	0.000000e+00	--	False	False
jet_leptonic	gamma0_log_parab	turn-over-energy	lorentz-factor*	2.581035e+03	1.000000e+00	1.000000e+09	False	False
jet_leptonic	s	LE_spectral_slope		2.195949e+00	-1.000000e+01	1.000000e+01	False	False
jet_leptonic	r	spectral_curvature		1.773625e-01	-1.500000e+01	1.500000e+01	False	False
jet_leptonic	R	region_size	cm	1.666152e+16	1.000000e+03	1.000000e+30	False	False
jet_leptonic	R_H	region_position	cm	1.000000e+17	0.000000e+00	--	False	True
jet_leptonic	B	magnetic_field	gauss	1.158370e-02	0.000000e+00	--	False	False
jet_leptonic	NH_cold_to_rel_e	cold_p_to_rel_e_ratio		1.000000e+00	0.000000e+00	--	False	True
jet_leptonic	beam_obj	beaming		4.369859e+01	1.000000e-04	--	False	False
jet_leptonic	z_cosm(M)	redshift		3.360000e-02	0.000000e+00	--	False	True
Franceschini_2008	scale_factor	scale_factor		1.000000e+00	0.000000e+00	--	False	True
Franceschini_2008	z_cosm(L,jet_leptonic)	redshift		--	--	--	False	True

converged=True
calls=365
mesg=

Fit parameters

Parameter	Best-fit value	Approximate error
EBL corrected.nuFnu_p_host	-9.96775	± 0.0308451
EBL corrected.gmin	285.076	± 135.655
EBL corrected.gmax	1.93856e+06	± 0
EBL corrected.N	6.15085	± 2.28928
EBL corrected.gamma0_log_parab	2581.04	± 0
EBL corrected.s	2.19595	± 0.105538
EBL corrected.r	0.177363	± 0.0322156
EBL corrected.R_sh	1.66615e+16	± 0
EBL corrected.B	0.0115837	± 0.00276367
EBL corrected.beam_obj	43.6986	± 6.26993

Summary (10)

Method

levmar

Statistic

chi2

Final statistic

13.9904

Number of evaluations

361

Reduced statistic

0.66621

Probability (Q-value)

0.870011

Initial statistic

147.741

Δ statistic

133.75

Number of data points

Degrees of freedom

dof=21
chisq=13.990405, chisq/red=0.666210 null hypothesis sig=0.870011

best fit pars

Table length=16

model name	name	val	bestfit val	err +	err -	start val	fit range min	fit range max	frozen
host_galaxy	nuFnu_p_host	-9.967750e+00	-9.967750e+00	3.084511e-02	--	-9.972939e+00	-1.219011e+01	-8.190114e+00	False
host_galaxy	nu_scale	-2.290459e-03	--	--	--	-2.290459e-03	-2.908697e-03	2.908697e-03	True
jet_leptonic	gmin	2.850763e+02	2.850763e+02	1.356548e+02	--	4.703917e+02	1.000000e+00	1.000000e+09	False
jet_leptonic	gmax	1.938561e+06	1.938561e+06	0.000000e+00	--	2.121873e+06	1.000000e+05	1.000000e+07	False
jet_leptonic	N	6.150849e+00	6.150849e+00	2.289278e+00	--	5.583988e+00	0.000000e+00	--	False
jet_leptonic	gamma0_log_parab	2.581035e+03	2.581035e+03	0.000000e+00	--	6.673207e+03	1.000000e+00	1.000000e+09	False
jet_leptonic	s	2.195949e+00	2.195949e+00	1.055377e-01	--	2.251647e+00	-1.000000e+01	1.000000e+01	False
jet_leptonic	r	1.773625e-01	1.773625e-01	3.221555e-02	--	2.794602e-01	-1.500000e+01	1.500000e+01	False
jet_leptonic	R	1.666152e+16	1.666152e+16	0.000000e+00	--	1.216622e+16	3.162278e+15	3.162278e+17	False
jet_leptonic	R_H	1.000000e+17	--	--	--	1.000000e+17	0.000000e+00	--	True
jet_leptonic	B	1.158370e-02	1.158370e-02	2.763668e-03	--	5.050000e-02	0.000000e+00	--	False
jet_leptonic	NH_cold_to_rel_e	1.000000e+00	--	--	--	1.000000e+00	0.000000e+00	--	True
jet_leptonic	beam_obj	4.369859e+01	4.369859e+01	6.269930e+00	--	2.500000e+01	5.000000e+00	5.000000e+01	False
jet_leptonic	z_cosm(M)	3.360000e-02	--	--	--	3.360000e-02	0.000000e+00	--	True
Franceschini_2008	scale_factor	1.000000e+00	--	--	--	1.000000e+00	0.000000e+00	--	True
Franceschini_2008	z_cosm(L,jet_leptonic)	3.360000e-02	--	--	--	--	0.000000e+00	--	True

-------------------------------------------------------------------------

================================================================================

composite_model.set_nu_grid(1E6,1E30,200)
composite_model.eval()
p=composite_model.plot_model(sed_data=sed_data)

../../../../_images/sherpa-plugin-jetset-interface_28_0.png

Using the sherpa_fitter you can access all the sherpa fetarues https://sherpa.readthedocs.io/en/latest/fit/index.html

model_minimizer_lsb.minimizer.sherpa_fitter.est_errors()

WARNING: hard minimum hit for parameter EBL corrected.gmax
WARNING: hard maximum hit for parameter EBL corrected.gmax
WARNING: hard minimum hit for parameter EBL corrected.B
WARNING: hard maximum hit for parameter EBL corrected.B

covariance 1σ (68.2689%) bounds

Parameter	Best-fit value	Lower Bound	Upper Bound
EBL corrected.nuFnu_p_host	-9.96775	-0.0306989	0.0306989
EBL corrected.gmin	285.076	-29.8952	29.8952
EBL corrected.gmax	1.93856e+06	-----	-----
EBL corrected.N	6.15085	-2.27219	2.27219
EBL corrected.gamma0_log_parab	2581.04	-663.942	663.942
EBL corrected.s	2.19595	-0.0280905	0.0280905
EBL corrected.r	0.177363	-0.018658	0.018658
EBL corrected.R_sh	1.66615e+16	-5.22449e+15	5.22449e+15
EBL corrected.B	0.0115837	-----	-----
EBL corrected.beam_obj	43.6986	-6.99752	6.99752

Summary (2)

Fitting Method