FRF Single Forecast Workflow

Version 1.0

The purpose of this workflow is to genreate a simple workflow of FRF and save the results that can be used for comparison and analysis It contains all the code snippets to be used for generate a blind test or forecast workflow Here are some guide lines: - (!) : Parameters are expected in the following box - Playground : Area to test and modify the result or generate alternatif runs

frf requierement needs to be installed - including frf lib dev version (pip install -e .)

### Module Import

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import logging
import os
import time
import pprint

from importlib import reload


%load_ext autoreload
%autoreload 2

from frflib.forecast_methods import forecast_wf
from frflib.data_class import input_data
from frflib.plots.forecast_plots import plot_group_forecast
from frflib.plots.forecast_plots import *
#from frflib.fieldbox_lib.workflow_interactions import get_preset_wf
import datetime
from frflib.data_class.input_data import InputData
import inspect
import frflib
reload(forecast_wf)
ForecastWF = forecast_wf.ForecastWF
InputData = input_data.InputData

PATH_FRFLIB= os.path.dirname(inspect.getsourcefile(frflib))
PATH_TO_DATA= os.path.join(PATH_FRFLIB,"sample_data")

The data source needs to be set up in order to read the production data. The data set needs to be in the frf inputdata format

(!) Project Info:

# if you want to select a subset of the field data - you can filter as follow
category = 'reservoir'
value = 'Fluvial'

Data Import

project_id="01"
wf_name="dataset_01_simple_wf"

WF_PATH=os.path.join(PATH_TO_DATA,f"{wf_name}.hdf")
DATA_PATH=os.path.join(PATH_TO_DATA,f"dataset_{project_id}.hdf")

field_data = InputData.load_hdf(DATA_PATH)
wf = ForecastWF.load_from_hdf( WF_PATH, field_data)
field_data._compute_indicators()
wf.enrich_data()
field_data_enriched = wf.enriched_data

(!) Parameters

If the worklow is going to be applied on a subset of data. It can be extracted using the function subset or a well lsit

field_data = field_data.subset(category = category, filt_value = value)

Number of processors available :

nproc = 4

Workflow param:

Here are the setup for the workflow. It can be done either a blind test of forcast. The parameters are the following: - Forecast name - Forecast mode: - Blind Test: To be used in order to test a workflow on past data. The needed data are: - Blind test duration: in months - forecast_fluids - Forecast: - forecast_end_date - abandonned_wells - uptime_selection: - constant_value - period_for_avg_months - Simple contraints

general_forecast_params ={
 'forecast_name': 'ref_wf_liq_cst_fonp',    # forecast reference Name
 'forecast_end_date': "01/01/2040",         # "dd/mm/yyyy"
 'forecast_fluids': ['oil',  'water'],      # ['oil',  'water', 'gas', 'liquid']
 'forecast_mode': 'forecast',               # 'forecast_mode': 'forecast', blind_test

#     Blind test
 'blind_test_duration': 120,                #  if forecast_mode is blind test - blind test duration


#     Foreward forecast                     #  if forecast_mode is forward
 'abandonned_wells': 12,                    #  wells closed during the last x months will be
 'uptime_selection': 'constant_value',      #  uptime computation method: constant_value or
 'constant_value': 1.0,
 'period_for_avg_months':12,                #
 'short_term_forecast_adjustment': 0,       # period to adjust linearly the trend to the last value (in months)
                                            # some methods like dca and analog will have some additional adj capabilities

#  constraints   - well level
 'eco_cut_off': 50,                       # economic cut off
 'max_liq_rate_stbd': None,                 # max liquid rate
 'max_gas_rate_mscfd': None,                # max gas rate
 'max_liq_from_history': None,              # max liquid rate

# constraints fields - NOT YET IMPLEMENTED
 'field_max_liq_rate_stbd': None,
 'field_max_water_stbd': None,
 'field_max_oil_stbd': None,
 'field_max_gas_rate_mscfd': None,
}

Methods:

Given the selected fluids, here is the list of the available methods in frflib. This list evolves regularely with news added methods. For additional information in every methods, you can contact FRF Team.

Not implemented the function ….get_help(…) will add some documentation

list(ForecastWF.get_available_f_methods(general_forecast_params['forecast_fluids']).keys())

['dca_oil',
 'analog_wells_oil',
 'cgr_cst',
 'curve_type_oil',
 'fitlearn_forecaster_oil',
 'bsw_fit',
 'analog_wells_bsw',
 'log_fo_fit',
 'adaptable_fo_fit',
 'bsw_cst',
 'wgr_cst',
 'curve_type_bsw',
 'dummy_forecaster_bsw',
 'fitlearn_forecaster_bsw']

Here are some preset example Methods:

#get_preset_wf(general_forecast_params['forecast_fluids'])

(!)

or build your own. The methods list needs to be in the correct order of execution rate / ratio, Oil or liquid / water …

Selected_methods = ['dca_oil', 'analog_wells_oil', 'adaptable_fo_fit', 'fitlearn_forecaster_bsw']

Playground

In the following, it is a playground area to generate more than one forecast with different parater

# Method 1
wf_0_general_param = general_forecast_params.copy()
wf_0_general_param['forecast_name'] = 'dca_oil_anaolg'
Selected_methods = ['dca_oil','analog_wells_oil', 'log_fo_fit','analog_wells_bsw']
wf_0_methods_param = [ForecastWF.get_f_methods_default_param(method, field_data) for method in Selected_methods]
# Method 2
# wf_1_general_param = general_forecast_params.copy()
# wf_1_general_param['forecast_name'] = 'dca_oil_dummy_forecaster'
# Selected_methods = ['dca_oil','dummy_forecaster_oil', 'log_fo_fit','analog_wells_bsw']
# wf_1_methods_param = [ForecastWF.get_f_methods_default_param(method, field_data) for method in Selected_methods]
# # Method 3
# wf_2_general_param = general_forecast_params.copy()
# wf_2_general_param['forecast_name'] = 'dca_oil_fit_learn'
# Selected_methods = ['dca_oil','fitlearn_forecaster_oil', 'log_fo_fit','analog_wells_bsw']
# wf_2_methods_param = [ForecastWF.get_f_methods_default_param(method, field_data) for method in Selected_methods]

Here are the default parameters of the selected methods

pprint.pprint(wf_0_methods_param)

[{'b_max': 1.0,
  'b_min': 0.0,
  'd_max': 100000.0,
  'd_min': 10,
  'fluid': ['oil'],
  'forecaster_name': 'dca_oil',
  'max_fit_points': 1000,
  'methods': 'hyperbolic',
  'min_fit_points': 18,
  'r2_min': 0.4,
  'shift_trend': True},
 {'fit_flag': False,
  'fluid': ['oil'],
  'forecaster_name': 'analog_wells_oil',
  'groupby_category': 'reservoir',
  'learn_x': 'cum_active_days',
  'neighbors_num': 4},
 {'beta_max': 1.0,
  'beta_min': 0.0,
  'bsw_fit_threshold': 0.2,
  'fluid': ['water'],
  'forecaster_name': 'log_fo_fit',
  'kernel_size': 3,
  'max_fit_points': 999,
  'min_bsw': 0.3,
  'min_fit_points': 9},
 {'fit_flag': False,
  'fluid': ['water'],
  'forecaster_name': 'analog_wells_bsw',
  'groupby_category': 'reservoir',
  'learn_x': 'cum_active_days',
  'neighbors_num': 4}]

Edit default parameters if needed:

wf_0_methods_param[0]['b_max'] = 0.5
# wf_0_methods_param[1]['early_date_filter'] = '01/01/2000'
# wf_1_methods_param[1]['npi_to_avg_rate'] = 2500
# wf_1_methods_param[1]['dca_d'] = 400
# wf_2_methods_param[1]['groupby_var'] = 'reservoir'
# wf_2_methods_param[1]['early_date_filter'] = '01/01/2010'
# default_param_list[2]['min_bsw'] = 0.3
# default_param_list[2]['bsw_fit_threshold'] = 0.1
# default_param_list[1]['groupby_var'] = 'reservoir'
# default_param_list[2]['max_fit_points'] = 22
# default_param_list[3]['groupby_var'] = 'reservoir'
# datetime.strptime(wf_2_methods_param[1]['early_date_filter'], "%d/%m/%Y")

Summary of the workflow:

pprint.pprint(wf_0_general_param)

{'abandonned_wells': 12,
 'blind_test_duration': 120,
 'constant_value': 1.0,
 'eco_cut_off': 50,
 'field_max_gas_rate_mscfd': None,
 'field_max_liq_rate_stbd': None,
 'field_max_oil_stbd': None,
 'field_max_water_stbd': None,
 'forecast_end_date': '01/01/2040',
 'forecast_fluids': ['oil', 'water'],
 'forecast_mode': 'forecast',
 'forecast_name': 'dca_oil_anaolg',
 'max_gas_rate_mscfd': None,
 'max_liq_from_history': None,
 'max_liq_rate_stbd': None,
 'period_for_avg_months': 12,
 'short_term_forecast_adjustment': 0,
 'uptime_selection': 'constant_value'}

pprint.pprint(wf_0_methods_param)

[{'b_max': 0.5,
  'b_min': 0.0,
  'd_max': 100000.0,
  'd_min': 10,
  'fluid': ['oil'],
  'forecaster_name': 'dca_oil',
  'max_fit_points': 1000,
  'methods': 'hyperbolic',
  'min_fit_points': 18,
  'r2_min': 0.4,
  'shift_trend': True},
 {'fit_flag': False,
  'fluid': ['oil'],
  'forecaster_name': 'analog_wells_oil',
  'groupby_category': 'reservoir',
  'learn_x': 'cum_active_days',
  'neighbors_num': 4},
 {'beta_max': 1.0,
  'beta_min': 0.0,
  'bsw_fit_threshold': 0.2,
  'fluid': ['water'],
  'forecaster_name': 'log_fo_fit',
  'kernel_size': 3,
  'max_fit_points': 999,
  'min_bsw': 0.3,
  'min_fit_points': 9},
 {'fit_flag': False,
  'fluid': ['water'],
  'forecaster_name': 'analog_wells_bsw',
  'groupby_category': 'reservoir',
  'learn_x': 'cum_active_days',
  'neighbors_num': 4}]

Workflow

### Event handling for blind test

The worflow can be exrcuted. In case of blind test it is important to check that there is no intervention in the wells to be forecasted. For this to function, the field data needs to contains event information (in df_event) and taggued correctly in event group

(!) Optional: Define wells with events to be removed from

event_list = ['stimulation']

if general_forecast_params['forecast_mode']=='blind_test' and field_data.df_events is not None:
    wl = field_data.df_events.loc[field_data.df_events['event_group'].isin(event_list)].index.get_level_values(0).unique()
    field_data = field_data.subset(wl = wl)

Forecast execution

Here you can execute more than one forecast that you prepared in the playground area

wf_0 = ForecastWF.execute(field_data,wf_0_general_param, wf_0_methods_param, nproc=1)
# wf_1 = ForecastWF.execute(field_data,wf_1_general_param, wf_1_methods_param, nproc=nproc)
# wf_2 = ForecastWF.execute(field_data,wf_2_general_param, wf_2_methods_param, nproc=nproc)

Results:

This section allows to represent the results at well and field scale You can visualize all the fluid available and all the wells. For this you need to select the workflow to visualize

(!)

wf_list = [wf_0]
x_axis = 'date'  # Other options cumulative oil: oil_cum_mmbbl
plot_active_wells = 'all' # add in a secondary axis the plot of the active wells
                                   # possible values 'history_only' / 'all' / 'no_history'

y_axis = 'oil_rate_stbd'
fig_field_oil_rate = plot_group_forecast(wf_list,
                                         x_axis=x_axis,
                                         y_axis =y_axis,
                                         plot_active_wells=plot_active_wells)
fig_field_oil_rate.show()

y_axis = 'water_rate_stbd'
fig_field_water_rate = plot_group_forecast(wf_list,
                                         x_axis=x_axis, y_axis =y_axis, plot_active_wells=plot_active_wells)
fig_field_water_rate.show()

Other variable possible to vizualise: - Rates: liq_rate_stbd, gas_rate_mscfd - Potential: pot_oil_rate_stbd, pot_water_rate_stbd - Cumulative: oil_cum_mmbbl, … - Ratios: bsw_vv, gor_vv

Playground:

You can visualize for a subset of well either through list or from the results

wl = list(wf_0.dict_wlist_fmethod['oil']['dca_oil']) # well teated by the forecaster dca_oil
# wl = [...,...]

y_axis = 'bsw_vv'
fig_field_oil_rate = plot_group_forecast(wf_list,
                                         x_axis=x_axis,
                                         y_axis =y_axis,
                                         well_list=wl,
                                         plot_active_wells=plot_active_wells)
fig_field_oil_rate.show()

y_axis = 'liq_rate_stbd'
fig_field_oil_rate = plot_group_forecast(wf_list,
                                         x_axis=x_axis,
                                         y_axis =y_axis,
                                         well_list=wl,
                                         plot_active_wells=plot_active_wells)
fig_field_oil_rate.show()

y_axis = 'pot_water_rate_stbd'
fig = plot_group_forecast(wf_list, x_axis=x_axis,y_axis =y_axis,
                                         well_list=wl,
                                         plot_active_wells=plot_active_wells)
fig.show()

Additional plots

The following plots need to specify the fluid (oil / water / liquid / gas) that you want to plot

(!)

fluid = 'oil'

Well number per method:

Shows the number of wells per forecast for a given fluid. You

fig_method_wells = wf_0.plot_well_num_per_fmethod(fluid)
fig_method_wells.show()

Blind test plots:

if general_forecast_params['forecast_mode'] == 'blind_test':
    fig_bt_wells = wf_0.plot_blind_test(fluid, well_list = None)
    fig_bt_wells.show()

if general_forecast_params['forecast_mode'] == 'blind_test':
    fig_bt_error = wf_0.plot_bt_error( fluid,well_list = None,sort = True, error = 'absolute')
    fig_bt_error.show()

Export for web app:

from frflib.utils.dataframe_func import df_fast_loc

from frflib.plots.utils import (get_bar_data_single, get_count_per_method_single, get_result_summary_single,
                                get_bt_dict, get_bar_bt, get_bar_data_single, get_single_summary, get_scatter_bt)


is_new_wells = False
is_mult = False
is_blindtest = False

dict_single_summary = get_single_summary(wf_0, well_list = None)

result_dict = {}
result_dict['meta_data']  = {'is_new_wells': is_new_wells,'is_mult': is_mult,'is_blindtest': is_blindtest}

result_dict['well_count'] = get_count_per_method_single(dict_single_summary)
result_dict['result_summary'] =get_result_summary_single(dict_single_summary)

if is_blindtest:
    bt_dict = get_bt_dict(wf_0, well_list = None)
    result_dict['bt_scatter'] = get_scatter_bt(bt_dict)
    result_dict['bt_bar'] = get_bar_bt(bt_dict)
else:
    result_dict['fw_bar'] = get_bar_data_single(dict_single_summary)

import json
def export_json(dict_plot, filename):
    with open(f"sample_data/{filename}", "w") as outfile:
        json.dump(dict_plot, outfile, default='convert')

#export_json(result_dict, 'for_front_single_fw.json')

Well by well results

wl = wf_0.dict_wlist_fmethod['oil']['analog_wells_oil']

plot_wells(wf_list ,y_axis_list = ['oil_rate_stbd'],x_axis='cum_active_days', well_number = 20, static_info = ['reservoir'], well_list=wl)

Extract results and parameters:

df = wf_0.get_result_summary(include_stat=True)
df.head()

	method	y_var	parameters	score	method_category	oil_uer_mmbbl	hist_oil_cum_mmbbl	rem_oil_uer_mmbbl	production_name	wellbore	...	last_water_rate_stbd	last_liq_rate_stbd	last_gas_rate_mscfd	last_bsw_vv	last_gor_vv	init_oil_rate_stbd	init_gor_vv	Cum_Prod_EHO_15	date_BSW_15	Months_BSW15
wellname
well_0	0	0	0	0.000000	0	0.000000	0.000000	0.000000	0	0	...	0.712533	27.676957	6245.300000	0.025745	41250.201701	107.241327	10685.999999	0.000000	0	0.0
well_1	analog_wells_oil	[pot_oil_rate_stbd]	Index(['well_133', 'well_50', 'well_122', 'wel...	0.162067	analog_wells	0.657545	0.568821	0.088724	JD28	JD28	...	12.000000	302.900000	106600.000000	0.039617	65264.558267	337.664158	10006.748923	0.000000	0	0.0
well_10	dca_oil	[pot_oil_rate_stbd]	{'qi': 335.6906287532635, 'd': 5480.2717592188...	0.880272	fit	0.118524	0.118524	0.000000	SA29	SA29_st1	...	0.225806	3.193543	903.224391	0.070707	54204.347821	247.698681	13824.383960	0.081248	2013-09-01 00:00:00	25.0
well_100	closed	[pot_oil_rate_stbd, pot_water_rate_stbd]	{}	1.000000	closed_wells	0.132563	0.132563	0.000000	RA08	RA08	...	0.695987	0.726795	4.296455	0.957611	24837.864236	179.388475	13675.558928	0.000000	2009-03-01 00:00:00	0.0
well_101	analog_wells_oil	[pot_oil_rate_stbd]	Index(['well_80', 'well_112', 'well_133', 'wel...	0.218708	analog_wells	0.507425	0.239892	0.267533	JB32	JB32_st1	...	0.000000	398.133333	121466.666700	0.000000	54336.604172	384.116215	15025.127954	0.000000	0	0.0

5 rows × 63 columns

Play ground

This a free area to test a analyse the result. To fully use the frflib capactities dont hesitate to contact FRF team

# example to access to a given forecast for additional analysis
forecaster = wf_0.dict_forecast_methods['oil']['dca_oil']

save worflow results

workflow_to_export = wf_0

(!) export parameters:

include_date = True
save_hdf = False

# csv export - Not fully implemented
save_csv_results = False
save_parameters = False
csv_file_format = 'frf' # phaser (Not yet implemented)

f_period = str(int(workflow_to_export.wf_param['forecast_period']/12))+"y"
file_name = f"bt_{f_period}_{workflow_to_export.name}" if workflow_to_export.wf_param['forecast_mode'] == 'blind_test' else f'fw_{f_period}_{workflow_to_export.name}'# file_name = "test"

if include_date:
    DATE = datetime.datetime.today().strftime("%Y-%m-%d")
    EXPORT_NAME = f"{DATE}_{file_name}"
else:
    EXPORT_NAME = file_name

print(EXPORT_NAME)

2024-07-29_fw_21y_dca_oil_anaolg

if save_hdf:
    EXPORT_NAME_hdf = f"{DATA_EXPORT_PATH + EXPORT_NAME}.hdf"
    workflow_to_export.save_to_hdf(EXPORT_NAME_hdf)

if save_csv_results:
    EXPORT_NAME_results_csv = f"{DATA_EXPORT_PATH + EXPORT_NAME}_data.csv"
    workflow_to_export.get_all_forecast().to_csv(EXPORT_NAME_results_csv)


if save_parameters:
    df = workflow_to_export.get_result_summary(include_stat=True)
    EXPORT_NAME_summary = f"{DATA_EXPORT_PATH + EXPORT_NAME}_summary.csv"
    df.to_csv(EXPORT_NAME_summary)

print('data exported')
print(EXPORT_NAME)

data exported
2024-07-29_fw_21y_dca_oil_anaolg

Create sample dataset for debugging:

Here is an area to create subset sampel for possible debugging

# field_data_short = copy.deepcopy(field_data)
# all_wells = list(field_data.df_dynamic.index.get_level_values(0).unique())
# # wl = set(['SR-113', 'SR-231'] + all_wells[0:10])
# field_data_short.df_dynamic = field_data_short.df_dynamic.loc[list(wl)]
# field_data_short.df_static = field_data_short.df_static.loc[list(wl)]

# field_data_short.save_hdf('sample_data/bug_wells_for_petrocedeno.hdf')