PORTFOLIO OF PROJECTS - Customer Churn using Survival Analysis (Part 2)

1 Business Understanding

Customer churn is a term that refers to the event that an existing customer (or client) of a company decides to separate from the company. This could mean that the customer switches to buying from a competitor, or simply, that the customer stops doing business with the company.

1.1 Modeling customer churn

It is important for digitally enabled companies to track customer churn. To model this aspect of customer behavior the first approach that comes to mind is supervised learning. A dataset will contain some features of customers and a target variable: Whether a customer has churned or not. A classification model could be helpful to identify customers that might churn some or other time. But will this be useful? Eventually, all customers will churn at some future time. What might be much more useful is to predict the lifetime of a customer, in other words, how long a customer will stay engaged with the company.

When predicting the lifetime of a customer, we change from a classification to a regression model as we need to predict a numerical value (lifetime). This regression approach, however, has a serious drawback. Most customers will not have churned, at a specific point in time. This means for all these datapoints we do not have a target value. We can choose to eliminate these datapoints from the dataset, but they represent the larger part of it! In addition, the fact that all these customers are still engaged with the company represent valuable information. It will be beneficial to have a method that can handle this complication. Such a method is that of Survival Analysis.

Survival analysis focuses on a unique kind of outcome variable: the time until an event occurs. In our present situation this event is the ‘death’ or churn of a customer. This will allow us to predict the time until a specific customer churns. For all customers in the dataset who are still active, we simply do not know when they will churn, and we simply flag their time-to-churn as censored. A customer record may also be censored in other cases, e.g. when a customer’s data are lost, or when a customer engaged with the company earlier than the date range of the dataset.

1.2 Survival and Censoring Times

We assume that for each customer, there is a

Survival time, \(T\)

This is the time at which the event of interest occurs. Survival time is also referred to as event time or failure time. In our present case this refers to the duration from engagement with the company until the cancellation of subscription, say (i.e. churn.

Censoring time \(C\)

This is time at which censoring occurs. This could be the time at which the last assessment of the cusomer is undertaken, or the time at which the dataset is compiled (given the customer is still engaged).

In our dataset, we will have two outcome variables:

Duration \(Y\)

We observe a duration which could be either a survival time \(T\), or else a censoring time \(C\). In fact, we observe the random variable \(Y = min(T, C)\). If \(T<C\) we observe the survival time \(T\). Alternatively, if \(T>C\) we observe the censoring time \(C\).

Event/Status indicator \(\delta\)

In addition to observing a duration, we also observe a status or event indicator,

\[ \delta = \begin{cases} 1 \text{ if } T \leq C\\ 0 \text{ if } T > C \end{cases} \]

When we observe a survival time \(\delta=1\), and if we instead observe a censoring time \(\delta=0\).

Dataset

The dataset consists of a set of datapoints \((x_1, y_1, \delta_1), ..., (x_i, y_i, \delta_i), ..., (x_n, y_n, \delta_n)\) where

\(x_i\) is the feature vector
\(y_i\) is the duration, either a survival time or a censoring time
\(\delta_i\) is the status/event indicator

This means we actually have two outcome variables: \(y\) and \(\delta\). The \(x\)’s are the usual predictor points.

1.3 Other survival analysis terminology

Usual statistical functions

The survival time \(T\) is a random variable and the usual statistical properties apply. First there is the cumulative distribution function (cdf) of \(T\):

\(F(t) = Pr\{T \leq t\}\)

Next, there is the probability density function (pdf) of \(T\):

\(f(t)\)

which is also known as the event density function, i.e. the rate of events per unit time.

The usual relationships exist between \(F(t)\) and \(f(t)\):

\(F(t) = \int_{-\infty}^{t}f(u)du\)

\(f(t) = \frac{dF(t)}{dt}\)

Survival function/probability

\(S(t) = Pr\{T > t\}\)

Survival probability in \(t\) is the probability that the survival time, \(T\) exceeds \(t\), i.e. the probability to survive beyond \(t\).

Note that

\(1 - F(t) = S(t)\), and

\(f(t) = -\frac{dS(t)}{dt}\)

Survival function estimate

When we estimate the survival probability with numerical techniques, we usually refer to the survival function estimate

\(\hat{S}(t)\)

Hazard function

The hazard function expresses the conditional probability that the event will occur within \(t, t+dt\), given that it has not occurred before

\(h(t) = \lim_{\Delta t \to 0} \frac{Pr\{t < T \leq t+\Delta t | T > t \}}{\Delta t} = \frac{f(t)}{S(t)} = -\frac{d}{dt}S(t)\)

\(h(t)\) is the event rate in the instant after time \(t\), given survival past that time.

Cumulative hazard function

\(H(t) = \int_{0}^{t}h(u)du\)

Some relationships

\(S(t) = \text{exp}[-\int_{0}^{t}h(u)du]\)

\(S(t) = \text{exp}[-H(t)]\)

\(h(t) = -[\frac{\frac{dS(t)}{dt}}{S(t)}]\)

1.4 The goal of survival analysis

The goal of survival analysis may be simply stated as to:

Estimate the time to the event of interest, \(T_j\) (survival time), for a new instance \(j\) with feature predictors denoted by \(X_j\).

2 Data Understanding

!pip install lifelines
!pip install scikit-survival

Requirement already satisfied: lifelines in /usr/local/lib/python3.7/dist-packages (0.27.0)
Requirement already satisfied: matplotlib>=3.0 in /usr/local/lib/python3.7/dist-packages (from lifelines) (3.2.2)
Requirement already satisfied: numpy>=1.14.0 in /usr/local/lib/python3.7/dist-packages (from lifelines) (1.21.5)
Requirement already satisfied: autograd-gamma>=0.3 in /usr/local/lib/python3.7/dist-packages (from lifelines) (0.5.0)
Requirement already satisfied: pandas>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from lifelines) (1.3.5)
Requirement already satisfied: formulaic>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from lifelines) (0.3.3)
Requirement already satisfied: scipy>=1.2.0 in /usr/local/lib/python3.7/dist-packages (from lifelines) (1.7.3)
Requirement already satisfied: autograd>=1.3 in /usr/local/lib/python3.7/dist-packages (from lifelines) (1.3)
Requirement already satisfied: future>=0.15.2 in /usr/local/lib/python3.7/dist-packages (from autograd>=1.3->lifelines) (0.16.0)
Requirement already satisfied: astor<0.8.0,>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from formulaic>=0.2.2->lifelines) (0.7.1)
Requirement already satisfied: wrapt>=1.0 in /usr/local/lib/python3.7/dist-packages (from formulaic>=0.2.2->lifelines) (1.14.0)
Requirement already satisfied: interface-meta<2.0.0,>=1.2.0 in /usr/local/lib/python3.7/dist-packages (from formulaic>=0.2.2->lifelines) (1.3.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0->lifelines) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0->lifelines) (1.4.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0->lifelines) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0->lifelines) (3.0.7)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib>=3.0->lifelines) (3.10.0.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.0->lifelines) (2018.9)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib>=3.0->lifelines) (1.15.0)
Requirement already satisfied: scikit-survival in /usr/local/lib/python3.7/dist-packages (0.17.1)
Requirement already satisfied: pandas>=0.25 in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (1.3.5)
Requirement already satisfied: numexpr in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (2.8.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (1.21.5)
Requirement already satisfied: scipy!=1.3.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (1.7.3)
Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (1.1.0)
Requirement already satisfied: osqp!=0.6.0,!=0.6.1 in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (0.6.2.post0)
Requirement already satisfied: scikit-learn<1.1,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (1.0.2)
Requirement already satisfied: ecos in /usr/local/lib/python3.7/dist-packages (from scikit-survival) (2.0.10)
Requirement already satisfied: qdldl in /usr/local/lib/python3.7/dist-packages (from osqp!=0.6.0,!=0.6.1->scikit-survival) (0.1.5.post0)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25->scikit-survival) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.25->scikit-survival) (2018.9)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=0.25->scikit-survival) (1.15.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn<1.1,>=1.0.0->scikit-survival) (3.1.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from numexpr->scikit-survival) (21.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->numexpr->scikit-survival) (3.0.7)

import warnings
warnings.filterwarnings('ignore') #6
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
from matplotlib import pyplot as plt
import seaborn as sns #7
import pandas as pd
pd.set_option('display.max_columns', None)
import numpy as np
import lifelines
import sksurv

plt.style.use('fivethirtyeight') #6
plt.rcParams['figure.figsize'] = [14, 8] #6
plt.rcParams['lines.linewidth'] = 2.5 #6

print('pandas:', pd.__version__)
print('lifelines:', lifelines.__version__)
print('numpy:', np.__version__)
print('scikit-survival:', sksurv.__version__)

pandas: 1.3.5
lifelines: 0.27.0
numpy: 1.21.5
scikit-survival: 0.17.1

Pointers related to the dataset:

https://github.com/treselle-systems/customer_churn_analysis https://www.kaggle.com/datasets/blastchar/telco-customer-churn

Other customer churn datasets:

https://community.ibm.com/community/user/businessanalytics/blogs/steven-macko/2019/07/11/telco-customer-churn-1113

df = pd.read_csv('https://raw.githubusercontent.com/'
                         'treselle-systems/customer_churn_analysis/'
                         'master/WA_Fn-UseC_-Telco-Customer-Churn.csv')
# df.head(10)
df

	customerID	gender	SeniorCitizen	Partner	Dependents	tenure	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	TotalCharges	Churn
0	7590-VHVEG	Female	0	Yes	No	1	No	No phone service	DSL	No	Yes	No	No	No	No	Month-to-month	Yes	Electronic check	29.85	29.85	No
1	5575-GNVDE	Male	0	No	No	34	Yes	No	DSL	Yes	No	Yes	No	No	No	One year	No	Mailed check	56.95	1889.5	No
2	3668-QPYBK	Male	0	No	No	2	Yes	No	DSL	Yes	Yes	No	No	No	No	Month-to-month	Yes	Mailed check	53.85	108.15	Yes
3	7795-CFOCW	Male	0	No	No	45	No	No phone service	DSL	Yes	No	Yes	Yes	No	No	One year	No	Bank transfer (automatic)	42.30	1840.75	No
4	9237-HQITU	Female	0	No	No	2	Yes	No	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Electronic check	70.70	151.65	Yes
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
7038	6840-RESVB	Male	0	Yes	Yes	24	Yes	Yes	DSL	Yes	No	Yes	Yes	Yes	Yes	One year	Yes	Mailed check	84.80	1990.5	No
7039	2234-XADUH	Female	0	Yes	Yes	72	Yes	Yes	Fiber optic	No	Yes	Yes	No	Yes	Yes	One year	Yes	Credit card (automatic)	103.20	7362.9	No
7040	4801-JZAZL	Female	0	Yes	Yes	11	No	No phone service	DSL	Yes	No	No	No	No	No	Month-to-month	Yes	Electronic check	29.60	346.45	No
7041	8361-LTMKD	Male	1	Yes	No	4	Yes	Yes	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Mailed check	74.40	306.6	Yes
7042	3186-AJIEK	Male	0	No	No	66	Yes	No	Fiber optic	Yes	No	Yes	Yes	Yes	Yes	Two year	Yes	Bank transfer (automatic)	105.65	6844.5	No

7043 rows × 21 columns

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 7043 entries, 0 to 7042
Data columns (total 21 columns):
 #   Column            Non-Null Count  Dtype  
---  ------            --------------  -----  
 0   customerID        7043 non-null   object 
 1   gender            7043 non-null   object 
 2   SeniorCitizen     7043 non-null   int64  
 3   Partner           7043 non-null   object 
 4   Dependents        7043 non-null   object 
 5   tenure            7043 non-null   int64  
 6   PhoneService      7043 non-null   object 
 7   MultipleLines     7043 non-null   object 
 8   InternetService   7043 non-null   object 
 9   OnlineSecurity    7043 non-null   object 
 10  OnlineBackup      7043 non-null   object 
 11  DeviceProtection  7043 non-null   object 
 12  TechSupport       7043 non-null   object 
 13  StreamingTV       7043 non-null   object 
 14  StreamingMovies   7043 non-null   object 
 15  Contract          7043 non-null   object 
 16  PaperlessBilling  7043 non-null   object 
 17  PaymentMethod     7043 non-null   object 
 18  MonthlyCharges    7043 non-null   float64
 19  TotalCharges      7043 non-null   object 
 20  Churn             7043 non-null   object 
dtypes: float64(1), int64(2), object(18)
memory usage: 1.1+ MB

# 
# need to fix TotalCharges for better understanding of this feature
df[df['TotalCharges']==' ']

	customerID	gender	Partner	Dependents	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	Churn
488	4472-LVYGI	Female	Yes	Yes	No	No phone service	DSL	Yes	No	Yes	Yes	Yes	No	Two year	Yes	Bank transfer (automatic)	52.55	No
753	3115-CZMZD	Male	No	Yes	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	20.25	No
936	5709-LVOEQ	Female	Yes	Yes	Yes	No	DSL	Yes	Yes	Yes	No	Yes	Yes	Two year	No	Mailed check	80.85	No
1082	4367-NUYAO	Male	Yes	Yes	Yes	Yes	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	25.75	No
1340	1371-DWPAZ	Female	Yes	Yes	No	No phone service	DSL	Yes	Yes	Yes	Yes	Yes	No	Two year	No	Credit card (automatic)	56.05	No
3331	7644-OMVMY	Male	Yes	Yes	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	19.85	No
3826	3213-VVOLG	Male	Yes	Yes	Yes	Yes	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	25.35	No
4380	2520-SGTTA	Female	Yes	Yes	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	20.00	No
5218	2923-ARZLG	Male	Yes	Yes	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	One year	Yes	Mailed check	19.70	No
6670	4075-WKNIU	Female	Yes	Yes	Yes	Yes	DSL	No	Yes	Yes	Yes	Yes	No	Two year	No	Mailed check	73.35	No
6754	2775-SEFEE	Male	No	Yes	Yes	Yes	DSL	Yes	Yes	No	Yes	No	No	Two year	Yes	Bank transfer (automatic)	61.90	No

df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
df

	customerID	gender	SeniorCitizen	Partner	Dependents	tenure	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	TotalCharges	Churn
0	7590-VHVEG	Female	0	Yes	No	1	No	No phone service	DSL	No	Yes	No	No	No	No	Month-to-month	Yes	Electronic check	29.85	29.85	No
1	5575-GNVDE	Male	0	No	No	34	Yes	No	DSL	Yes	No	Yes	No	No	No	One year	No	Mailed check	56.95	1889.50	No
2	3668-QPYBK	Male	0	No	No	2	Yes	No	DSL	Yes	Yes	No	No	No	No	Month-to-month	Yes	Mailed check	53.85	108.15	Yes
3	7795-CFOCW	Male	0	No	No	45	No	No phone service	DSL	Yes	No	Yes	Yes	No	No	One year	No	Bank transfer (automatic)	42.30	1840.75	No
4	9237-HQITU	Female	0	No	No	2	Yes	No	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Electronic check	70.70	151.65	Yes
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
7038	6840-RESVB	Male	0	Yes	Yes	24	Yes	Yes	DSL	Yes	No	Yes	Yes	Yes	Yes	One year	Yes	Mailed check	84.80	1990.50	No
7039	2234-XADUH	Female	0	Yes	Yes	72	Yes	Yes	Fiber optic	No	Yes	Yes	No	Yes	Yes	One year	Yes	Credit card (automatic)	103.20	7362.90	No
7040	4801-JZAZL	Female	0	Yes	Yes	11	No	No phone service	DSL	Yes	No	No	No	No	No	Month-to-month	Yes	Electronic check	29.60	346.45	No
7041	8361-LTMKD	Male	1	Yes	No	4	Yes	Yes	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Mailed check	74.40	306.60	Yes
7042	3186-AJIEK	Male	0	No	No	66	Yes	No	Fiber optic	Yes	No	Yes	Yes	Yes	Yes	Two year	Yes	Bank transfer (automatic)	105.65	6844.50	No

7043 rows × 21 columns

df['TotalCharges'].isnull().sum()

df_desc = df.describe(include="all").T
df_desc

	count	unique	top	freq	mean	std	min	25%	50%	75%	max
customerID	7043	7043	7590-VHVEG	1	NaN	NaN	NaN	NaN	NaN	NaN	NaN
gender	7043	2	Male	3555	NaN	NaN	NaN	NaN	NaN	NaN	NaN
SeniorCitizen	7043.0	NaN	NaN	NaN	0.162147	0.368612	0.0	0.0	0.0	0.0	1.0
Partner	7043	2	No	3641	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Dependents	7043	2	No	4933	NaN	NaN	NaN	NaN	NaN	NaN	NaN
tenure	7043.0	NaN	NaN	NaN	32.371149	24.559481	0.0	9.0	29.0	55.0	72.0
PhoneService	7043	2	Yes	6361	NaN	NaN	NaN	NaN	NaN	NaN	NaN
MultipleLines	7043	3	No	3390	NaN	NaN	NaN	NaN	NaN	NaN	NaN
InternetService	7043	3	Fiber optic	3096	NaN	NaN	NaN	NaN	NaN	NaN	NaN
OnlineSecurity	7043	3	No	3498	NaN	NaN	NaN	NaN	NaN	NaN	NaN
OnlineBackup	7043	3	No	3088	NaN	NaN	NaN	NaN	NaN	NaN	NaN
DeviceProtection	7043	3	No	3095	NaN	NaN	NaN	NaN	NaN	NaN	NaN
TechSupport	7043	3	No	3473	NaN	NaN	NaN	NaN	NaN	NaN	NaN
StreamingTV	7043	3	No	2810	NaN	NaN	NaN	NaN	NaN	NaN	NaN
StreamingMovies	7043	3	No	2785	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Contract	7043	3	Month-to-month	3875	NaN	NaN	NaN	NaN	NaN	NaN	NaN
PaperlessBilling	7043	2	Yes	4171	NaN	NaN	NaN	NaN	NaN	NaN	NaN
PaymentMethod	7043	4	Electronic check	2365	NaN	NaN	NaN	NaN	NaN	NaN	NaN
MonthlyCharges	7043.0	NaN	NaN	NaN	64.761692	30.090047	18.25	35.5	70.35	89.85	118.75
TotalCharges	7032.0	NaN	NaN	NaN	2283.300441	2266.771362	18.8	401.45	1397.475	3794.7375	8684.8
Churn	7043	2	No	5174	NaN	NaN	NaN	NaN	NaN	NaN	NaN

df.nunique()

customerID          7043
gender                 2
SeniorCitizen          2
Partner                2
Dependents             2
tenure                73
PhoneService           2
MultipleLines          3
InternetService        3
OnlineSecurity         3
OnlineBackup           3
DeviceProtection       3
TechSupport            3
StreamingTV            3
StreamingMovies        3
Contract               3
PaperlessBilling       2
PaymentMethod          4
MonthlyCharges      1585
TotalCharges        6530
Churn                  2
dtype: int64

[print(f"{col}: {df[col].unique()}") for col in df]

customerID: ['7590-VHVEG' '5575-GNVDE' '3668-QPYBK' ... '4801-JZAZL' '8361-LTMKD'
 '3186-AJIEK']
gender: ['Female' 'Male']
SeniorCitizen: [0 1]
Partner: ['Yes' 'No']
Dependents: ['No' 'Yes']
tenure: [ 1 34  2 45  8 22 10 28 62 13 16 58 49 25 69 52 71 21 12 30 47 72 17 27
  5 46 11 70 63 43 15 60 18 66  9  3 31 50 64 56  7 42 35 48 29 65 38 68
 32 55 37 36 41  6  4 33 67 23 57 61 14 20 53 40 59 24 44 19 54 51 26  0
 39]
PhoneService: ['No' 'Yes']
MultipleLines: ['No phone service' 'No' 'Yes']
InternetService: ['DSL' 'Fiber optic' 'No']
OnlineSecurity: ['No' 'Yes' 'No internet service']
OnlineBackup: ['Yes' 'No' 'No internet service']
DeviceProtection: ['No' 'Yes' 'No internet service']
TechSupport: ['No' 'Yes' 'No internet service']
StreamingTV: ['No' 'Yes' 'No internet service']
StreamingMovies: ['No' 'Yes' 'No internet service']
Contract: ['Month-to-month' 'One year' 'Two year']
PaperlessBilling: ['Yes' 'No']
PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'
 'Credit card (automatic)']
MonthlyCharges: [29.85 56.95 53.85 ... 63.1  44.2  78.7 ]
TotalCharges: [  29.85 1889.5   108.15 ...  346.45  306.6  6844.5 ]
Churn: ['No' 'Yes']

[None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None]

# 
# check missing values
df.isnull().sum()

customerID           0
gender               0
SeniorCitizen        0
Partner              0
Dependents           0
tenure               0
PhoneService         0
MultipleLines        0
InternetService      0
OnlineSecurity       0
OnlineBackup         0
DeviceProtection     0
TechSupport          0
StreamingTV          0
StreamingMovies      0
Contract             0
PaperlessBilling     0
PaymentMethod        0
MonthlyCharges       0
TotalCharges        11
Churn                0
dtype: int64

response variables

The role of response variables is played by ‘tenure’ for \(y\) and ‘Churn’ for \(\delta\).

plt.style.use('fivethirtyeight') #6
plt.rcParams['figure.figsize'] = [14, 8] #6
plt.rcParams['lines.linewidth'] = 2.5 #6

df['tenure'].hist(color='skyblue');

df['Churn'].value_counts(normalize=True).plot.bar(color=['lightgreen','red']);

df['MonthlyCharges'].hist(color='skyblue');

df['TotalCharges'].hist(color='skyblue');

Mth = sns.kdeplot(df['MonthlyCharges'][(df["Churn"]=='No')],
                color="Red", shade = True)
Mth = sns.kdeplot(df['MonthlyCharges'][(df["Churn"]=='Yes')],
                ax=Mth, color="Blue", shade= True)
Mth.legend(["No Churn", "Churn"], loc='upper right')
Mth.set_ylabel('Density')
Mth.set_xlabel('Monthly Charges')
Mth.set_title('Monthly charges by churn');

mypal = [
    'lightblue',    
    'lightpink',
    'lightcyan',
    'lightgreen',
    'khaki',
    'gold',
    'wheat',
    'peachpuff',
    'mistyrose',
    'sandybrown',
    'violet',
]

# df['gender'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['gender'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['SeniorCitizen'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['SeniorCitizen'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['Partner'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['Partner'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['Dependents'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['Dependents'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['Contract'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['Contract'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['PaperlessBilling'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['PaperlessBilling'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['PaymentMethod'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['PaymentMethod'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['PhoneService'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['PhoneService'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['MultipleLines'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['MultipleLines'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['InternetService'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['InternetService'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['OnlineSecurity'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['OnlineSecurity'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['OnlineBackup'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['OnlineBackup'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['DeviceProtection'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['DeviceProtection'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['TechSupport'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['TechSupport'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['StreamingTV'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['StreamingTV'], df['Churn']).plot.bar(color=mypal, stacked=True);

# df['StreamingMovies'].value_counts(normalize=True).plot.bar(color=mypal);
pd.crosstab(df['StreamingMovies'], df['Churn']).plot.bar(color=mypal, stacked=True);

# 
# sns.heatmap(df.corr(), cmap="Paired")
# Generate a mask for the upper triangle #7
corr = df.corr()
mask = np.triu(np.ones_like(corr, dtype=np.bool))
# Generate a custom diverging colormap
cmap = sns.diverging_palette(220, 10, as_cmap=True)
# Plot correlation matrix
sns.heatmap(corr, cmap=cmap, linewidths=.5, mask=mask);

# 
# inspect life times of a sample of customers
df_life = df[['tenure','Churn']]
df_life = df_life.replace({'Churn': {'No': False, 'Yes': True}})
df_life

	tenure	Churn
0	1	False
1	34	False
2	2	True
3	45	False
4	2	True
...	...	...
7038	24	False
7039	72	False
7040	11	False
7041	4	True
7042	66	False

7043 rows × 2 columns

from lifelines.plotting import plot_lifetimes
n = 40
sample = df_life.sample(n, replace=False)
durations = sample['tenure']
status = sample['Churn']

plt.figure(figsize=(18, 15));
plot_lifetimes(
    durations=durations, event_observed=status,
#     entry=None,
#     left_truncated=False,
    sort_by_duration=False,
    event_observed_color='red', #'#A60628',
    event_censored_color='blue', #'#348ABD',
)
plt.xlabel('Days subscribed', fontsize=16);
plt.ylabel('Customer ID', fontsize=16);
plt.title('Customer subscription lifelines (random sample)', fontsize=20)
plt.tight_layout()

#8 How many right-censored samples?
print(f'Number of samples: {len(df)}')
print(f'Number of right censored samples: {len(df.query("Churn == False"))}')
print(f'Percentage of right censored samples: {100*len(df.query("Churn == False"))/len(df):.1f}%')

Number of samples: 7043
Number of right censored samples: 0
Percentage of right censored samples: 0.0%

3 Data Preparation

df['customerID'].duplicated().any()

False

df = df.set_index('customerID')
df

	gender	SeniorCitizen	Partner	Dependents	tenure	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	TotalCharges	Churn
customerID
7590-VHVEG	Female	0	Yes	No	1	No	No phone service	DSL	No	Yes	No	No	No	No	Month-to-month	Yes	Electronic check	29.85	29.85	No
5575-GNVDE	Male	0	No	No	34	Yes	No	DSL	Yes	No	Yes	No	No	No	One year	No	Mailed check	56.95	1889.50	No
3668-QPYBK	Male	0	No	No	2	Yes	No	DSL	Yes	Yes	No	No	No	No	Month-to-month	Yes	Mailed check	53.85	108.15	Yes
7795-CFOCW	Male	0	No	No	45	No	No phone service	DSL	Yes	No	Yes	Yes	No	No	One year	No	Bank transfer (automatic)	42.30	1840.75	No
9237-HQITU	Female	0	No	No	2	Yes	No	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Electronic check	70.70	151.65	Yes
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
6840-RESVB	Male	0	Yes	Yes	24	Yes	Yes	DSL	Yes	No	Yes	Yes	Yes	Yes	One year	Yes	Mailed check	84.80	1990.50	No
2234-XADUH	Female	0	Yes	Yes	72	Yes	Yes	Fiber optic	No	Yes	Yes	No	Yes	Yes	One year	Yes	Credit card (automatic)	103.20	7362.90	No
4801-JZAZL	Female	0	Yes	Yes	11	No	No phone service	DSL	Yes	No	No	No	No	No	Month-to-month	Yes	Electronic check	29.60	346.45	No
8361-LTMKD	Male	1	Yes	No	4	Yes	Yes	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Mailed check	74.40	306.60	Yes
3186-AJIEK	Male	0	No	No	66	Yes	No	Fiber optic	Yes	No	Yes	Yes	Yes	Yes	Two year	Yes	Bank transfer (automatic)	105.65	6844.50	No

7043 rows × 20 columns

3.1 Event/Censor times/durations

df['tenure'] = df['tenure'].astype(float)

3.2 Status indicator

df['Churn'] = (df['Churn']=='Yes')

3.3 Numericals

df[df['TotalCharges'].isnull()]

	gender	SeniorCitizen	Partner	Dependents	tenure	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	TotalCharges	Churn
customerID
4472-LVYGI	Female	0	Yes	Yes	0.0	No	No phone service	DSL	Yes	No	Yes	Yes	Yes	No	Two year	Yes	Bank transfer (automatic)	52.55	NaN	False
3115-CZMZD	Male	0	No	Yes	0.0	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	20.25	NaN	False
5709-LVOEQ	Female	0	Yes	Yes	0.0	Yes	No	DSL	Yes	Yes	Yes	No	Yes	Yes	Two year	No	Mailed check	80.85	NaN	False
4367-NUYAO	Male	0	Yes	Yes	0.0	Yes	Yes	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	25.75	NaN	False
1371-DWPAZ	Female	0	Yes	Yes	0.0	No	No phone service	DSL	Yes	Yes	Yes	Yes	Yes	No	Two year	No	Credit card (automatic)	56.05	NaN	False
7644-OMVMY	Male	0	Yes	Yes	0.0	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	19.85	NaN	False
3213-VVOLG	Male	0	Yes	Yes	0.0	Yes	Yes	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	25.35	NaN	False
2520-SGTTA	Female	0	Yes	Yes	0.0	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	Two year	No	Mailed check	20.00	NaN	False
2923-ARZLG	Male	0	Yes	Yes	0.0	Yes	No	No	No internet service	No internet service	No internet service	No internet service	No internet service	No internet service	One year	Yes	Mailed check	19.70	NaN	False
4075-WKNIU	Female	0	Yes	Yes	0.0	Yes	Yes	DSL	No	Yes	Yes	Yes	Yes	No	Two year	No	Mailed check	73.35	NaN	False
2775-SEFEE	Male	0	No	Yes	0.0	Yes	Yes	DSL	Yes	Yes	No	Yes	No	No	Two year	Yes	Bank transfer (automatic)	61.90	NaN	False

# df['Price'] = df['Price'].astype(float)
df['TotalCharges'] = df['TotalCharges'].fillna(0.0)

df[df['TotalCharges'].isnull()]

	gender	SeniorCitizen	Partner	Dependents	tenure	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	TotalCharges	Churn
customerID

3.4 Categoricals

# 
# Categoricals will be prepared later

4 Modeling

4.1 Survival Probability for the average customer

First we will use the Kaplan-Meier non-parametric model to estimate the survival probability of an average customer, i.e. not for an individual customer.

df

	gender	SeniorCitizen	Partner	Dependents	tenure	PhoneService	MultipleLines	InternetService	OnlineSecurity	OnlineBackup	DeviceProtection	TechSupport	StreamingTV	StreamingMovies	Contract	PaperlessBilling	PaymentMethod	MonthlyCharges	TotalCharges	Churn
customerID
7590-VHVEG	Female	0	Yes	No	1.0	No	No phone service	DSL	No	Yes	No	No	No	No	Month-to-month	Yes	Electronic check	29.85	29.85	False
5575-GNVDE	Male	0	No	No	34.0	Yes	No	DSL	Yes	No	Yes	No	No	No	One year	No	Mailed check	56.95	1889.50	False
3668-QPYBK	Male	0	No	No	2.0	Yes	No	DSL	Yes	Yes	No	No	No	No	Month-to-month	Yes	Mailed check	53.85	108.15	True
7795-CFOCW	Male	0	No	No	45.0	No	No phone service	DSL	Yes	No	Yes	Yes	No	No	One year	No	Bank transfer (automatic)	42.30	1840.75	False
9237-HQITU	Female	0	No	No	2.0	Yes	No	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Electronic check	70.70	151.65	True
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
6840-RESVB	Male	0	Yes	Yes	24.0	Yes	Yes	DSL	Yes	No	Yes	Yes	Yes	Yes	One year	Yes	Mailed check	84.80	1990.50	False
2234-XADUH	Female	0	Yes	Yes	72.0	Yes	Yes	Fiber optic	No	Yes	Yes	No	Yes	Yes	One year	Yes	Credit card (automatic)	103.20	7362.90	False
4801-JZAZL	Female	0	Yes	Yes	11.0	No	No phone service	DSL	Yes	No	No	No	No	No	Month-to-month	Yes	Electronic check	29.60	346.45	False
8361-LTMKD	Male	1	Yes	No	4.0	Yes	Yes	Fiber optic	No	No	No	No	No	No	Month-to-month	Yes	Mailed check	74.40	306.60	True
3186-AJIEK	Male	0	No	No	66.0	Yes	No	Fiber optic	Yes	No	Yes	Yes	Yes	Yes	Two year	Yes	Bank transfer (automatic)	105.65	6844.50	False

7043 rows × 20 columns

from sksurv.nonparametric import kaplan_meier_estimator
durations = df['tenure'].values
churns = df['Churn'].values

# kmf = lifelines.KaplanMeierFitter()
# kmf.fit(durations=durations, event_observed=churns, label='Estimate for Average Customer')
time, survival_prob = kaplan_meier_estimator(churns, durations)

time

array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12.,
       13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.,
       26., 27., 28., 29., 30., 31., 32., 33., 34., 35., 36., 37., 38.,
       39., 40., 41., 42., 43., 44., 45., 46., 47., 48., 49., 50., 51.,
       52., 53., 54., 55., 56., 57., 58., 59., 60., 61., 62., 63., 64.,
       65., 66., 67., 68., 69., 70., 71., 72.])

survival_prob

array([1.        , 0.94596132, 0.92783494, 0.91372452, 0.90104451,
       0.89111052, 0.88482624, 0.87671295, 0.86993299, 0.86239401,
       0.85491516, 0.84969096, 0.84319955, 0.83660256, 0.83237374,
       0.82578174, 0.82072556, 0.8159762 , 0.81153149, 0.80795314,
       0.80451991, 0.80123616, 0.79596229, 0.79338315, 0.7887364 ,
       0.78400357, 0.7808709 , 0.77810863, 0.7755171 , 0.77223657,
       0.76867992, 0.76506473, 0.76070865, 0.7574498 , 0.75461291,
       0.75100693, 0.74854541, 0.74480393, 0.7415061 , 0.7378978 ,
       0.73449734, 0.73076758, 0.72696008, 0.72280127, 0.72110522,
       0.71937871, 0.71584802, 0.71162116, 0.70884018, 0.70409137,
       0.70085119, 0.69818762, 0.6954456 , 0.69047051, 0.68571369,
       0.68232094, 0.67843307, 0.67519279, 0.67057967, 0.66710291,
       0.66440391, 0.66062621, 0.65814452, 0.65605683, 0.65384974,
       0.64854347, 0.64038175, 0.63359804, 0.62685764, 0.6201353 ,
       0.60965683, 0.602781  , 0.59279015])

plt.step(time, survival_prob, where="post")
plt.title('Kaplan-Meier Customer Survival Probability - All Customers')
plt.ylabel("est. probability of survival $\hat{S}(t)$")
plt.xlabel("time $t$")
plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red']);

As seen from the chart, the survival probability \(S(t)\) declines with tenure. Customer retention efforts can be costly and a company will lose profitability if such measures are applied without discretion. Let’s say we want to have our customer retention procedure only kick in when \(S(t)\) falls below 70%. From the chart it appears that this will have to happen when the tenure exceeds 51 months. This model represents the average customer which means customer retention will always happen when a customer’s tenure exceeds 51 months.

4.2 Survival Probability for the average customer based on segments

The previous approach is unfocused and may lead to a lot of waste. We can improve the effectiveness somewhat by studying the survival functions for different segments of the customer population. Segmentation can be performed based on the classes in categorical predictors. Let’s demonstrate this approach by making use of the ‘InternetService’ predictor.

# 
#create Kaplan-Meier curve plots segmented by categoricals
def plot_categorical_survival(dft, feature, duration='tenure', event='Churn', ax=None):
    for cat in dft[feature].unique():
        idxs = dft[feature]==cat
        time, survival_prob = kaplan_meier_estimator(dft[idxs][event], dft[idxs][duration])
        #plt.legend(cat)
        #plt.step(time, survival_prob, where="post", label='treat={}'.format(cat))
        plt.step(time, survival_prob, where="post", label='treat={}'.format(cat))
    return dft[feature].unique()

# fig_pmt, ax_pmt = plt.subplots(figsize=(12,7))
# plot_categorical_survival(dft=df, feature='PaymentMethod', ax=ax_pmt)
cats = plot_categorical_survival(dft=df, feature='PaymentMethod')
# ax_pmt.set_title('Customer Churn by Payment Method')
plt.title('Customer Churn by Payment Method')
# ax_pmt.set_xlabel('tenure (Months)')
plt.xlabel('tenure (Months)')
# ax_pmt.set_ylabel('Survival Probability')
plt.ylabel('Survival Probability')
plt.legend(cats)
plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red'])
plt.show()

cats = plot_categorical_survival(dft=df, feature='Contract')
plt.title('Customer Churn by Contract Type')
plt.xlabel('tenure (Months)')
plt.ylabel('Survival Probability')
plt.legend(cats)
plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red'])
plt.show()

cats = plot_categorical_survival(dft=df, feature='Dependents')
plt.title('Customer Churn, Dependents vs. No Dependents')
plt.xlabel('tenure (Months)')
plt.ylabel('Survival Probability')
plt.legend(cats)
plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red'])
plt.show()

cats = plot_categorical_survival(dft=df, feature='InternetService')
plt.title('Customer Churn by InternetService')
plt.xlabel('tenure (Months)')
plt.ylabel('Survival Probability')
plt.legend(cats)
plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red'])
plt.show()

From the chart we see that we can be more effective by only activating the retention procedure for fiber optic customers (when their tenure exceeds about 26 months).

4.3 Estimate Hazard

Before we can use another approach to model the average customer, we need to make use of the concept of hazard. The hazard function or hazard rate was defined above. It is the event rate in the instant after \(t\), given survival past \(t\). In the context of customer churn the hazard is an important indicator and needs to be as low as possible. If it goes up it indicates something has changed between the company and a customer. Such a situation calls for intervention.

On a more technical level, why should we care about the hazard function? Firstly, it is related to the survival function \(S(t)\). Secondly, an important technique for modeling survival time as a function of covariates (in this case customer features) depends on the hazard function. This kind of technique will be necessary to analyze an individual customer, rather than the whole population.

4.4 Nelson-Aalen estimator

The Nelson-Aalen estimator estimates the cumulative hazard rate. Like the Kaplan-Meier estimator, it is non-parametric.

4.4.1 Cumulative hazard function

from sksurv.nonparametric import nelson_aalen_estimator
time, cum_hazard = nelson_aalen_estimator(
    event=df['Churn'].values, 
    time=df['tenure'].values)

plt.step(time, cum_hazard, where="post")
plt.title('Nelson-Aalen Customer Cumulative Hazard - All Customers')
plt.ylabel("est. probability of survival $\hat{S}(t)$")
plt.xlabel("time $t$")
# plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red'])

Text(0.5, 0, 'time $t$')

# fig, ax = plt.subplots(figsize=(10,7))
var = 'InternetService'
legend_names = []
for name, dft in df.groupby(var):
    idxs = df[var]==name
    time, cum_hazard = nelson_aalen_estimator(
        event=dft['Churn'][idxs], 
        time=dft['tenure'][idxs])    
    plt.step(time, cum_hazard, where="post")
    legend_names.append(name)
ax.set_title(var)
ax.set_xlabel('tenure (Months)')
ax.set_ylabel('cumulative hazard rate')
ax.legend(legend_names)
# plt.hlines(y=0.7, xmin=0, xmax=70, linestyles='dotted', colors=['red'])
plt.show()

4.4.2 Hazard function

4.5 Survival Regression Model

So far, we have only looked at analyzing groups of customers. Now we are going to setup a model for an individual customer. This will increase the effectiveness of the customer retention mechanism and allow us to model the survival time as a function of the covariates (i.e. the customer features). First, we need to do some pre-processing before we can fit these kinds of models.

4.5.1 Split the data

from sksurv.datasets import get_x_y
X_df, y = get_x_y(df, attr_labels=['Churn','tenure'], pos_label=True)

type(X_df)

pandas.core.frame.DataFrame

array([(False,  1.), (False, 34.), ( True,  2.), ..., (False, 11.),
       ( True,  4.), (False, 66.)],
      dtype=[('Churn', '?'), ('tenure', '<f8')])

type(y)

numpy.ndarray

# without converted y
from sklearn.model_selection import train_test_split
X_trn_df, X_tst_df, y_trn, y_tst = train_test_split(
    X_df, y, 
    test_size=0.2, 
    random_state=42, 
    shuffle=True, 
    #to not end up with test data only containing censored event times
    stratify=y['Churn'])

# 
# convert y_trn & y_tst to a df (nice for consistency, but not required) #9
y_trn_df = pd.DataFrame(
    data={
        'tenure': [y_trn[i][1] for i in range(len(y_trn))], 
        'Churn': [y_trn[i][0] for i in range(len(y_trn))]})
y_tst_df = pd.DataFrame(
    data={
        'tenure': [y_tst[i][1] for i in range(len(y_tst))], 
        'Churn': [y_tst[i][0] for i in range(len(y_tst))]})
y_trn_df

	tenure	Churn
0	35.0	False
1	15.0	False
2	13.0	False
3	26.0	False
4	1.0	False
...	...	...
5629	71.0	False
5630	2.0	True
5631	25.0	True
5632	24.0	False
5633	6.0	False

5634 rows × 2 columns

type(X_trn_df), type(y_trn_df)

(pandas.core.frame.DataFrame, pandas.core.frame.DataFrame)

print(f'training samples: {len(y_trn)}')
print(f'testing samples: {len(y_tst)}')
# print(f'training samples: {len(y_trn_df)}')
# print(f'testing samples: {len(y_tst_df)}')

training samples: 5634
testing samples: 1409

df.columns

Index(['gender', 'SeniorCitizen', 'Partner', 'Dependents', 'tenure',
       'PhoneService', 'MultipleLines', 'InternetService', 'OnlineSecurity',
       'OnlineBackup', 'DeviceProtection', 'TechSupport', 'StreamingTV',
       'StreamingMovies', 'Contract', 'PaperlessBilling', 'PaymentMethod',
       'MonthlyCharges', 'TotalCharges', 'Churn'],
      dtype='object')

nums = ['MonthlyCharges', 'TotalCharges']
cats = ['gender', 'SeniorCitizen', 'Partner', 'Dependents', 'PhoneService',
        'MultipleLines', 'InternetService', 'OnlineSecurity', 'OnlineBackup',
        'DeviceProtection', 'TechSupport', 'StreamingTV', 'StreamingMovies',
        'Contract', 'PaperlessBilling', 'PaymentMethod']
# tenure, Churn is y
print(nums)
print(cats)

['MonthlyCharges', 'TotalCharges']
['gender', 'SeniorCitizen', 'Partner', 'Dependents', 'PhoneService', 'MultipleLines', 'InternetService', 'OnlineSecurity', 'OnlineBackup', 'DeviceProtection', 'TechSupport', 'StreamingTV', 'StreamingMovies', 'Contract', 'PaperlessBilling', 'PaymentMethod']

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OrdinalEncoder
from sklearn.preprocessing import OneHotEncoder
from sklearn.preprocessing import StandardScaler
preprocessor = ColumnTransformer(
    transformers=[
        ('one-hot', OneHotEncoder(drop='first'), cats),
        ('std-sca', StandardScaler(), nums)],
    remainder='passthrough', sparse_threshold=0)
X_trn_prp = preprocessor.fit_transform(X_trn_df)
X_tst_prp = preprocessor.transform(X_tst_df)

type(X_trn_prp)

numpy.ndarray

# 
# https://johaupt.github.io/blog/columnTransformer_feature_names.html
import warnings
import sklearn
def get_feature_names(column_transformer):
    """Get feature names from all transformers.
    Returns
    -------
    feature_names : list of strings
        Names of the features produced by transform.
    """
    # Remove the internal helper function
    #check_is_fitted(column_transformer)

    # Turn lookup into function for better handling with pipeline later
    def get_names(trans):
        # >> Original get_feature_names() method
        if trans == 'drop' or (
                hasattr(column, '__len__') and not len(column)):
            return []
        if trans == 'passthrough':
            if hasattr(column_transformer, '_df_columns'):
                if ((not isinstance(column, slice))
                        and all(isinstance(col, str) for col in column)):
                    return column
                else:
                    return column_transformer._df_columns[column]
            else:
                indices = np.arange(column_transformer._n_features)
                return ['x%d' % i for i in indices[column]]
        if not hasattr(trans, 'get_feature_names'):
        # >>> Change: Return input column names if no method avaiable
            # Turn error into a warning
            warnings.warn("Transformer %s (type %s) does not provide get_feature_names. Will return input column names if available" % (str(name), type(trans).__name__))
            # For transformers without a get_features_names method, use the input names to the column transformer
            if column is None:
                return []
            else:
                return [name + "__" + f for f in column]
        return [name + "__" + f for f in trans.get_feature_names()]

    ### Start of processing
    feature_names = []
    # Allow transformers to be pipelines. Pipeline steps are named differently, so preprocessing is needed
    if type(column_transformer) == sklearn.pipeline.Pipeline:
        l_transformers = [(name, trans, None, None) for step, name, trans in column_transformer._iter()]
    else:
        # For column transformers, follow the original method
        l_transformers = list(column_transformer._iter(fitted=True))
    for name, trans, column, _ in l_transformers: 
        if type(trans) == sklearn.pipeline.Pipeline:
            # Recursive call on pipeline
            _names = get_feature_names(trans)
            # if pipeline has no transformer that returns names
            if len(_names)==0:
                _names = [name + "__" + f for f in column]
            feature_names.extend(_names)
        else:
            feature_names.extend(get_names(trans))
    return feature_names

get_feature_names(preprocessor)
# len(get_feature_names(preprocessor))

['one-hot__x0_Male',
 'one-hot__x1_1',
 'one-hot__x2_Yes',
 'one-hot__x3_Yes',
 'one-hot__x4_Yes',
 'one-hot__x5_No phone service',
 'one-hot__x5_Yes',
 'one-hot__x6_Fiber optic',
 'one-hot__x6_No',
 'one-hot__x7_No internet service',
 'one-hot__x7_Yes',
 'one-hot__x8_No internet service',
 'one-hot__x8_Yes',
 'one-hot__x9_No internet service',
 'one-hot__x9_Yes',
 'one-hot__x10_No internet service',
 'one-hot__x10_Yes',
 'one-hot__x11_No internet service',
 'one-hot__x11_Yes',
 'one-hot__x12_No internet service',
 'one-hot__x12_Yes',
 'one-hot__x13_One year',
 'one-hot__x13_Two year',
 'one-hot__x14_Yes',
 'one-hot__x15_Credit card (automatic)',
 'one-hot__x15_Electronic check',
 'one-hot__x15_Mailed check',
 'std-sca__MonthlyCharges',
 'std-sca__TotalCharges']

# 
# form X_trn_prp_df (handy when column names are required or a dataframe)
X_trn_prp_df = pd.DataFrame(X_trn_prp)
X_trn_prp_df.columns = get_feature_names(preprocessor)
X_trn_prp_df

	one-hot__x0_Male	one-hot__x1_1	one-hot__x2_Yes	one-hot__x3_Yes	one-hot__x4_Yes	one-hot__x5_No phone service	one-hot__x5_Yes	one-hot__x6_Fiber optic	one-hot__x6_No	one-hot__x7_No internet service	one-hot__x7_Yes	one-hot__x8_No internet service	one-hot__x8_Yes	one-hot__x9_No internet service	one-hot__x9_Yes	one-hot__x10_No internet service	one-hot__x10_Yes	one-hot__x11_No internet service	one-hot__x11_Yes	one-hot__x12_No internet service	one-hot__x12_Yes	one-hot__x13_One year	one-hot__x13_Two year	one-hot__x14_Yes	one-hot__x15_Credit card (automatic)	one-hot__x15_Electronic check	one-hot__x15_Mailed check	std-sca__MonthlyCharges	std-sca__TotalCharges
0	1.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	1.0	0.0	-0.521976	-0.262257
1	1.0	0.0	1.0	1.0	1.0	0.0	0.0	1.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.337478	-0.503635
2	1.0	0.0	1.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	1.0	-0.809013	-0.749883
3	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	1.0	1.0	0.0	0.0	0.284384	-0.172722
4	1.0	0.0	1.0	1.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	-0.676279	-0.989374
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
5629	0.0	0.0	1.0	0.0	1.0	0.0	1.0	1.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	0.0	1.470695	2.373129
5630	1.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	-0.626504	-0.973665
5631	0.0	1.0	0.0	0.0	1.0	0.0	1.0	1.0	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	0.0	0.0	1.0	1.256662	0.158344
5632	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	0.0	0.0	-1.477661	-0.797075
5633	1.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	1.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	-1.469365	-0.960962

5634 rows × 29 columns

# 
# form X_tst_prp_df (handy when column names are required or a dataframe)
X_tst_prp_df = pd.DataFrame(X_tst_prp)
X_tst_prp_df.columns = get_feature_names(preprocessor)
X_tst_prp_df

	one-hot__x0_Male	one-hot__x1_1	one-hot__x2_Yes	one-hot__x3_Yes	one-hot__x4_Yes	one-hot__x5_No phone service	one-hot__x5_Yes	one-hot__x6_Fiber optic	one-hot__x6_No	one-hot__x7_No internet service	one-hot__x7_Yes	one-hot__x8_No internet service	one-hot__x8_Yes	one-hot__x9_No internet service	one-hot__x9_Yes	one-hot__x10_No internet service	one-hot__x10_Yes	one-hot__x11_No internet service	one-hot__x11_Yes	one-hot__x12_No internet service	one-hot__x12_Yes	one-hot__x13_One year	one-hot__x13_Two year	one-hot__x14_Yes	one-hot__x15_Credit card (automatic)	one-hot__x15_Electronic check	one-hot__x15_Mailed check	std-sca__MonthlyCharges	std-sca__TotalCharges
0	1.0	0.0	1.0	1.0	1.0	0.0	1.0	1.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	1.0	1.0	0.0	0.0	1.629976	2.706828
1	0.0	1.0	0.0	0.0	1.0	0.0	1.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	1.0	0.0	0.0	1.168725	-0.610260
2	0.0	0.0	1.0	1.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	0.0	1.0	0.0	1.0	1.0	0.0	0.0	0.445324	0.400116
3	1.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.440347	-0.364451
4	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	1.0	1.0	0.0	0.0	0.588013	1.588421
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
1404	0.0	0.0	1.0	1.0	1.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	1.0	0.0	1.0	0.0	0.0	1.0	0.738999	0.897615
1405	1.0	0.0	1.0	1.0	1.0	0.0	0.0	0.0	1.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	1.0	1.0	0.0	0.0	-1.480980	-0.794815
1406	1.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	1.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	-1.469365	-0.967873
1407	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	1.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0	-1.500890	-0.547360
1408	0.0	0.0	1.0	1.0	1.0	0.0	1.0	0.0	0.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	1.0	1.0	0.0	0.0	0.0	0.183174	1.257728

1409 rows × 29 columns

4.5.2 Proportional Hazards Assumption

The proportional hazards assumption states

\(h(t|x_i) = h_0(t) \text{exp} (\sum_{j=1}^{p}x_{ij}\beta_j)\)

where \(h_0(t)\), an unspecified function, is known as the baseline hazard. It is the hazard function for an individual with features \(x_{i1} = ... = x_{ip} = 0\). The term proportional hazards derives from the fact that the hazard function for an individual with feature vector \(x_i\) is some unknown function \(h_0(t)\) times the factor \(\text{exp} (\sum_{j=1}^{p}x_{ij}\beta_j)\). The latter quantity is called the relative risk for the feature vector \(x_i = (x_{i1}, ..., x_{ip})^T\), relative to that for the feature vector \(x_i = (0, ..., 0)^T\).

The baseline hazard function \(h_0(t)\) can take on any form. The only assumption is that a one-unit increase in \(x_{ij}\) corresponds to an increase in \(h(t|x_i)\) by a factor of exp(\(\beta_j\)).

4.5.3 Cox Proportional Hazards Regression

We next make use of the Cox Proportional Hazards model.

from sksurv.linear_model import CoxPHSurvivalAnalysis
from sksurv.metrics import concordance_index_censored

cph = CoxPHSurvivalAnalysis(alpha=0.1)
cph.fit(X_trn_prp, y_trn)

CoxPHSurvivalAnalysis(alpha=0.1)

print(cph.score(X_tst_prp, y_tst))
# OR
ci_cph = concordance_index_censored(y_tst["Churn"], y_tst["tenure"], cph.predict(X_tst_prp))
cph.score(X_tst_prp, y_tst)
print(f'The c-index of Cox PH is given by {ci_cph[0]:.3f}')

0.9174183981986005
The c-index of Cox PH is given by 0.917

cph.score(X_trn_prp, y_trn) #just for interest sake

0.9304333852400903

len(cph.coef_), len(get_feature_names(preprocessor))

(29, 29)

# 
# pd.Series(cph.coef_, index=get_feature_names(preprocessor))
# pd.Series(cph.coef_.flatten(), index=get_feature_names(preprocessor))
pd.Series(cph.coef_, index=X_trn_prp_df.columns)

one-hot__x0_Male                       -0.003764
one-hot__x1_1                          -0.024939
one-hot__x2_Yes                        -0.145846
one-hot__x3_Yes                        -0.140579
one-hot__x4_Yes                         0.613211
one-hot__x5_No phone service           -0.613211
one-hot__x5_Yes                         0.196545
one-hot__x6_Fiber optic                 1.451681
one-hot__x6_No                         -0.390536
one-hot__x7_No internet service        -0.390536
one-hot__x7_Yes                        -0.118828
one-hot__x8_No internet service        -0.390536
one-hot__x8_Yes                         0.023259
one-hot__x9_No internet service        -0.390536
one-hot__x9_Yes                         0.209663
one-hot__x10_No internet service       -0.390536
one-hot__x10_Yes                        0.026421
one-hot__x11_No internet service       -0.390536
one-hot__x11_Yes                        0.478477
one-hot__x12_No internet service       -0.390536
one-hot__x12_Yes                        0.466215
one-hot__x13_One year                  -1.306728
one-hot__x13_Two year                  -3.731098
one-hot__x14_Yes                        0.164420
one-hot__x15_Credit card (automatic)    0.004613
one-hot__x15_Electronic check           0.387792
one-hot__x15_Mailed check               0.547358
std-sca__MonthlyCharges                -0.174482
std-sca__TotalCharges                  -3.549800
dtype: float64

4.5.3.1 Survival function

4.5.3.1.1 A validation customer

X_tst_prp[0]

array([1.        , 0.        , 1.        , 1.        , 1.        ,
       0.        , 1.        , 1.        , 0.        , 0.        ,
       1.        , 0.        , 1.        , 0.        , 1.        ,
       0.        , 1.        , 0.        , 1.        , 0.        ,
       1.        , 0.        , 1.        , 1.        , 1.        ,
       0.        , 0.        , 1.62997635, 2.70682752])

X_tst_prp[0].shape

(29,)

X_tst_prp[0].reshape(1, -1).shape

(1, 29)

S_cph = cph.predict_survival_function(X_tst_prp[0].reshape(1, -1))

len(S_cph)

S_cph = S_cph[0]

# 
# plt.step(S[0].x, S[0].y, where='post');
plt.step(S_cph.x, S_cph.y, where='post');

S = cph.predict_survival_function(X_tst_prp)

for curve in S[0:50]:
    #print(curve.x)
    plt.step(curve.x, curve.y, where='post')

4.5.3.2 Cumulative hazard function

H_cph = cph.predict_cumulative_hazard_function(X_tst_prp[0].reshape(1, -1))

H_cph = H_cph[0]

plt.step(H_cph.x, H_cph.y, where='post');

4.5.3.1.2 Some fictitious customers

# 
# show an example to build from
print(list(X_tst_prp[0]))

[1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.6299763471022684, 2.7068275180093]

# get_feature_names(preprocessor)

x_new = pd.DataFrame.from_dict({
    1: [0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, -1.30432, 0.123],
    2: [1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, -1.10193, 1.567],
    3: [0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, -1.49384, -1.546]
},
    #columns=data_x_numeric.columns, 
    orient='index')

x_new

	0	1	2	3	5	6	8	9	11	15	17	18	19	22	23	25	27	28
1	0	1	0	1	0	1	1	0	0	1	0	0	1	1	1	1	-1.30432	0.123
2	1	1	1	1	1	1	0	1	1	1	1	1	1	1	1	1	-1.10193	1.567
3	0	1	1	1	0	1	1	0	0	0	0	0	1	0	1	1	-1.49384	-1.546

import numpy as np
surv_funcs = cph.predict_survival_function(x_new)
time_points = np.arange(1, len(surv_funcs[0].x))
for i, surv_func in enumerate(surv_funcs):
    plt.step(time_points, surv_func(time_points), where="post",
             label="Customer %d" % (i + 1))
plt.ylabel("est. survival probability $\hat{S}(t)$")
plt.xlabel("time $t$")
plt.legend(loc="best");

4.5.4 Feature Selection: Which Variable is Most Predictive

The model above considered all available variables for prediction. Next, we want to investigate which single variable is the best risk predictor. Therefore, we fit a Cox model to each variable individually and record the c-index on the training set.

# 
# https://scikit-survival.readthedocs.io/en/stable/user_guide/00-introduction.html
def fit_and_score_features(X, y):
    n_features = X.shape[1]
    scores = np.empty(n_features)
    m = CoxPHSurvivalAnalysis()
    for j in range(n_features):
        Xj = X[:, j:j+1]
        m.fit(Xj, y)
        scores[j] = m.score(Xj, y)
    return scores

scores = fit_and_score_features(X_trn_prp, y_trn)
#pd.Series(scores, index=get_feature_names(preprocessor)).sort_values(ascending=False)
pd.Series(scores, index=X_trn_prp_df.columns).sort_values(ascending=False)

std-sca__TotalCharges                   0.868790
one-hot__x13_Two year                   0.658932
one-hot__x15_Electronic check           0.648528
one-hot__x2_Yes                         0.628519
one-hot__x6_Fiber optic                 0.612156
one-hot__x7_Yes                         0.610834
one-hot__x10_Yes                        0.610263
one-hot__x13_One year                   0.604419
one-hot__x8_Yes                         0.597975
one-hot__x9_Yes                         0.590484
one-hot__x3_Yes                         0.585407
one-hot__x15_Credit card (automatic)    0.579490
one-hot__x14_Yes                        0.574733
one-hot__x7_No internet service         0.569734
one-hot__x11_No internet service        0.569734
one-hot__x8_No internet service         0.569734
one-hot__x12_No internet service        0.569734
one-hot__x10_No internet service        0.569734
one-hot__x9_No internet service         0.569734
one-hot__x6_No                          0.569734
one-hot__x5_Yes                         0.550919
one-hot__x12_Yes                        0.537151
one-hot__x1_1                           0.536699
one-hot__x11_Yes                        0.531166
std-sca__MonthlyCharges                 0.511292
one-hot__x0_Male                        0.505213
one-hot__x5_No phone service            0.502687
one-hot__x4_Yes                         0.502687
one-hot__x15_Mailed check               0.493576
dtype: float64

# 
# https://www.pythonfixing.com/2021/10/fixed-scikit-learn-pipeline-sparse.html
from sklearn.base import TransformerMixin
class DenseTransformer(TransformerMixin):
    def fit(self, X, y=None, **fit_params):
        return self

    def transform(self, X, y=None, **fit_params):
        return X.todense()

from sklearn.feature_selection import SelectKBest
from sklearn.pipeline import Pipeline
pipe = Pipeline([('encode', OneHotEncoder()),
                 ('to_dense', DenseTransformer()), 
                 ('select', SelectKBest(fit_and_score_features, k=3)),
                 ('model', CoxPHSurvivalAnalysis())])

%%time
from sklearn.model_selection import GridSearchCV, KFold

param_grid = {'select__k': np.arange(1, X_trn_prp.shape[1] + 1)}
cv = KFold(n_splits=3, random_state=1, shuffle=True)
gcv = GridSearchCV(pipe, param_grid, return_train_score=True, cv=cv, verbose=10, n_jobs=-1)
# gcv.fit(data_x, data_y)
gcv.fit(X_df, y)

results = pd.DataFrame(gcv.cv_results_).sort_values(by='mean_test_score', ascending=False)
results.loc[:, ~results.columns.str.endswith("_time")]

Fitting 3 folds for each of 29 candidates, totalling 87 fits
[CV 1/3; 1/29] START select__k=1................................................
[CV 1/3; 1/29] END .select__k=1;, score=(train=nan, test=nan) total time= 4.1min
[CV 2/3; 1/29] START select__k=1................................................
[CV 2/3; 1/29] END .select__k=1;, score=(train=nan, test=nan) total time= 2.8min
[CV 3/3; 1/29] START select__k=1................................................
[CV 3/3; 1/29] END .select__k=1;, score=(train=nan, test=nan) total time= 6.8min
[CV 1/3; 2/29] START select__k=2................................................
[CV 1/3; 2/29] END .select__k=2;, score=(train=nan, test=nan) total time= 4.4min
[CV 2/3; 2/29] START select__k=2................................................
[CV 2/3; 2/29] END .select__k=2;, score=(train=nan, test=nan) total time= 2.8min
[CV 3/3; 2/29] START select__k=2................................................
[CV 3/3; 2/29] END .select__k=2;, score=(train=nan, test=nan) total time= 7.4min
[CV 1/3; 3/29] START select__k=3................................................
[CV 1/3; 3/29] END .select__k=3;, score=(train=nan, test=nan) total time= 3.6min
[CV 2/3; 3/29] START select__k=3................................................
[CV 2/3; 3/29] END .select__k=3;, score=(train=nan, test=nan) total time= 2.9min
[CV 3/3; 3/29] START select__k=3................................................
[CV 3/3; 3/29] END .select__k=3;, score=(train=nan, test=nan) total time= 6.8min
[CV 1/3; 4/29] START select__k=4................................................
[CV 1/3; 4/29] END .select__k=4;, score=(train=nan, test=nan) total time= 3.9min
[CV 2/3; 4/29] START select__k=4................................................
[CV 2/3; 4/29] END .select__k=4;, score=(train=nan, test=nan) total time= 3.3min
[CV 3/3; 4/29] START select__k=4................................................
[CV 3/3; 4/29] END .select__k=4;, score=(train=nan, test=nan) total time= 7.0min
[CV 1/3; 5/29] START select__k=5................................................
[CV 1/3; 5/29] END .select__k=5;, score=(train=nan, test=nan) total time= 3.4min
[CV 2/3; 5/29] START select__k=5................................................
[CV 2/3; 5/29] END .select__k=5;, score=(train=nan, test=nan) total time= 3.1min
[CV 3/3; 5/29] START select__k=5................................................
[CV 3/3; 5/29] END .select__k=5;, score=(train=nan, test=nan) total time= 7.2min
[CV 1/3; 6/29] START select__k=6................................................
[CV 1/3; 6/29] END .select__k=6;, score=(train=nan, test=nan) total time= 3.1min
[CV 2/3; 6/29] START select__k=6................................................
[CV 2/3; 6/29] END .select__k=6;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 6/29] START select__k=6................................................
[CV 3/3; 6/29] END .select__k=6;, score=(train=nan, test=nan) total time= 6.0min
[CV 1/3; 7/29] START select__k=7................................................
[CV 1/3; 7/29] END .select__k=7;, score=(train=nan, test=nan) total time= 3.0min
[CV 2/3; 7/29] START select__k=7................................................
[CV 2/3; 7/29] END .select__k=7;, score=(train=nan, test=nan) total time= 2.6min
[CV 3/3; 7/29] START select__k=7................................................
[CV 3/3; 7/29] END .select__k=7;, score=(train=nan, test=nan) total time= 6.3min
[CV 1/3; 8/29] START select__k=8................................................
[CV 1/3; 8/29] END .select__k=8;, score=(train=nan, test=nan) total time= 3.4min
[CV 2/3; 8/29] START select__k=8................................................
[CV 2/3; 8/29] END .select__k=8;, score=(train=nan, test=nan) total time= 2.8min
[CV 3/3; 8/29] START select__k=8................................................
[CV 3/3; 8/29] END .select__k=8;, score=(train=nan, test=nan) total time= 6.4min
[CV 1/3; 9/29] START select__k=9................................................
[CV 1/3; 9/29] END .select__k=9;, score=(train=nan, test=nan) total time= 3.1min
[CV 2/3; 9/29] START select__k=9................................................
[CV 2/3; 9/29] END .select__k=9;, score=(train=nan, test=nan) total time= 2.5min
[CV 3/3; 9/29] START select__k=9................................................
[CV 3/3; 9/29] END .select__k=9;, score=(train=nan, test=nan) total time= 6.0min
[CV 1/3; 10/29] START select__k=10..............................................
[CV 1/3; 10/29] END select__k=10;, score=(train=nan, test=nan) total time= 3.0min
[CV 2/3; 10/29] START select__k=10..............................................
[CV 2/3; 10/29] END select__k=10;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 10/29] START select__k=10..............................................
[CV 3/3; 10/29] END select__k=10;, score=(train=nan, test=nan) total time= 6.0min
[CV 1/3; 11/29] START select__k=11..............................................
[CV 1/3; 11/29] END select__k=11;, score=(train=nan, test=nan) total time= 3.0min
[CV 2/3; 11/29] START select__k=11..............................................
[CV 2/3; 11/29] END select__k=11;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 11/29] START select__k=11..............................................
[CV 3/3; 11/29] END select__k=11;, score=(train=nan, test=nan) total time= 6.0min
[CV 1/3; 12/29] START select__k=12..............................................
[CV 1/3; 12/29] END select__k=12;, score=(train=nan, test=nan) total time= 3.0min
[CV 2/3; 12/29] START select__k=12..............................................
[CV 2/3; 12/29] END select__k=12;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 12/29] START select__k=12..............................................
[CV 3/3; 12/29] END select__k=12;, score=(train=nan, test=nan) total time= 6.9min
[CV 1/3; 13/29] START select__k=13..............................................
[CV 1/3; 13/29] END select__k=13;, score=(train=nan, test=nan) total time= 3.5min
[CV 2/3; 13/29] START select__k=13..............................................
[CV 2/3; 13/29] END select__k=13;, score=(train=nan, test=nan) total time= 2.7min
[CV 3/3; 13/29] START select__k=13..............................................
[CV 3/3; 13/29] END select__k=13;, score=(train=nan, test=nan) total time= 7.1min
[CV 1/3; 14/29] START select__k=14..............................................
[CV 1/3; 14/29] END select__k=14;, score=(train=nan, test=nan) total time= 3.5min
[CV 2/3; 14/29] START select__k=14..............................................
[CV 2/3; 14/29] END select__k=14;, score=(train=nan, test=nan) total time= 2.7min
[CV 3/3; 14/29] START select__k=14..............................................
[CV 3/3; 14/29] END select__k=14;, score=(train=nan, test=nan) total time= 6.7min
[CV 1/3; 15/29] START select__k=15..............................................
[CV 1/3; 15/29] END select__k=15;, score=(train=nan, test=nan) total time= 3.1min
[CV 2/3; 15/29] START select__k=15..............................................
[CV 2/3; 15/29] END select__k=15;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 15/29] START select__k=15..............................................
[CV 3/3; 15/29] END select__k=15;, score=(train=nan, test=nan) total time= 6.8min
[CV 1/3; 16/29] START select__k=16..............................................
[CV 1/3; 16/29] END select__k=16;, score=(train=nan, test=nan) total time= 3.5min
[CV 2/3; 16/29] START select__k=16..............................................
[CV 2/3; 16/29] END select__k=16;, score=(train=nan, test=nan) total time= 2.7min
[CV 3/3; 16/29] START select__k=16..............................................
[CV 3/3; 16/29] END select__k=16;, score=(train=nan, test=nan) total time= 7.0min
[CV 1/3; 17/29] START select__k=17..............................................
[CV 1/3; 17/29] END select__k=17;, score=(train=nan, test=nan) total time= 3.3min
[CV 2/3; 17/29] START select__k=17..............................................
[CV 2/3; 17/29] END select__k=17;, score=(train=nan, test=nan) total time= 2.7min
[CV 3/3; 17/29] START select__k=17..............................................
[CV 3/3; 17/29] END select__k=17;, score=(train=nan, test=nan) total time= 6.8min
[CV 1/3; 18/29] START select__k=18..............................................
[CV 1/3; 18/29] END select__k=18;, score=(train=nan, test=nan) total time= 3.4min
[CV 2/3; 18/29] START select__k=18..............................................
[CV 2/3; 18/29] END select__k=18;, score=(train=nan, test=nan) total time= 2.8min
[CV 3/3; 18/29] START select__k=18..............................................
[CV 3/3; 18/29] END select__k=18;, score=(train=nan, test=nan) total time= 7.5min
[CV 1/3; 19/29] START select__k=19..............................................
[CV 1/3; 19/29] END select__k=19;, score=(train=nan, test=nan) total time= 3.4min
[CV 2/3; 19/29] START select__k=19..............................................
[CV 2/3; 19/29] END select__k=19;, score=(train=nan, test=nan) total time= 2.6min
[CV 3/3; 19/29] START select__k=19..............................................
[CV 3/3; 19/29] END select__k=19;, score=(train=nan, test=nan) total time= 6.5min
[CV 1/3; 20/29] START select__k=20..............................................
[CV 1/3; 20/29] END select__k=20;, score=(train=nan, test=nan) total time= 3.7min
[CV 2/3; 20/29] START select__k=20..............................................
[CV 2/3; 20/29] END select__k=20;, score=(train=nan, test=nan) total time= 3.1min
[CV 3/3; 20/29] START select__k=20..............................................
[CV 3/3; 20/29] END select__k=20;, score=(train=nan, test=nan) total time= 6.2min
[CV 1/3; 21/29] START select__k=21..............................................
[CV 1/3; 21/29] END select__k=21;, score=(train=nan, test=nan) total time= 3.0min
[CV 2/3; 21/29] START select__k=21..............................................
[CV 2/3; 21/29] END select__k=21;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 21/29] START select__k=21..............................................
[CV 3/3; 21/29] END select__k=21;, score=(train=nan, test=nan) total time= 6.1min
[CV 1/3; 22/29] START select__k=22..............................................
[CV 1/3; 22/29] END select__k=22;, score=(train=nan, test=nan) total time= 3.1min
[CV 2/3; 22/29] START select__k=22..............................................
[CV 2/3; 22/29] END select__k=22;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 22/29] START select__k=22..............................................
[CV 3/3; 22/29] END select__k=22;, score=(train=nan, test=nan) total time= 6.0min
[CV 1/3; 23/29] START select__k=23..............................................
[CV 1/3; 23/29] END select__k=23;, score=(train=nan, test=nan) total time= 3.6min
[CV 2/3; 23/29] START select__k=23..............................................
[CV 2/3; 23/29] END select__k=23;, score=(train=nan, test=nan) total time= 2.8min
[CV 3/3; 23/29] START select__k=23..............................................
[CV 3/3; 23/29] END select__k=23;, score=(train=nan, test=nan) total time= 7.2min
[CV 1/3; 24/29] START select__k=24..............................................
[CV 1/3; 24/29] END select__k=24;, score=(train=nan, test=nan) total time= 3.4min
[CV 2/3; 24/29] START select__k=24..............................................
[CV 2/3; 24/29] END select__k=24;, score=(train=nan, test=nan) total time= 2.8min
[CV 3/3; 24/29] START select__k=24..............................................
[CV 3/3; 24/29] END select__k=24;, score=(train=nan, test=nan) total time= 7.5min
[CV 1/3; 25/29] START select__k=25..............................................
[CV 1/3; 25/29] END select__k=25;, score=(train=nan, test=nan) total time= 3.1min
[CV 2/3; 25/29] START select__k=25..............................................
[CV 2/3; 25/29] END select__k=25;, score=(train=nan, test=nan) total time= 2.4min
[CV 3/3; 25/29] START select__k=25..............................................
[CV 3/3; 25/29] END select__k=25;, score=(train=nan, test=nan) total time= 7.0min
[CV 1/3; 26/29] START select__k=26..............................................
[CV 1/3; 26/29] END select__k=26;, score=(train=nan, test=nan) total time= 3.3min
[CV 2/3; 26/29] START select__k=26..............................................
[CV 2/3; 26/29] END select__k=26;, score=(train=nan, test=nan) total time= 2.7min
[CV 3/3; 26/29] START select__k=26..............................................
[CV 3/3; 26/29] END select__k=26;, score=(train=nan, test=nan) total time= 7.0min
[CV 1/3; 27/29] START select__k=27..............................................
[CV 1/3; 27/29] END select__k=27;, score=(train=nan, test=nan) total time= 4.6min
[CV 2/3; 27/29] START select__k=27..............................................
[CV 2/3; 27/29] END select__k=27;, score=(train=nan, test=nan) total time= 2.7min
[CV 3/3; 27/29] START select__k=27..............................................
[CV 3/3; 27/29] END select__k=27;, score=(train=nan, test=nan) total time= 8.7min
[CV 1/3; 28/29] START select__k=28..............................................
[CV 1/3; 28/29] END select__k=28;, score=(train=nan, test=nan) total time= 4.1min
[CV 2/3; 28/29] START select__k=28..............................................
[CV 2/3; 28/29] END select__k=28;, score=(train=nan, test=nan) total time= 3.2min
[CV 3/3; 28/29] START select__k=28..............................................
[CV 3/3; 28/29] END select__k=28;, score=(train=nan, test=nan) total time= 8.2min
[CV 1/3; 29/29] START select__k=29..............................................
[CV 1/3; 29/29] END select__k=29;, score=(train=nan, test=nan) total time= 3.8min
[CV 2/3; 29/29] START select__k=29..............................................
[CV 2/3; 29/29] END select__k=29;, score=(train=nan, test=nan) total time= 3.1min
[CV 3/3; 29/29] START select__k=29..............................................
[CV 3/3; 29/29] END select__k=29;, score=(train=nan, test=nan) total time= 7.7min

LinAlgError: Matrix is singular.

pipe.set_params(**gcv.best_params_)
pipe.fit(X_df, y)

encoder, transformer, final_estimator = [s[1] for s in pipe.steps]
pd.Series(final_estimator.coef_, index=encoder.encoded_columns_[transformer.get_support()])

LinAlgError: Matrix is singular.

4.6 Penalized Cox Models

https://scikit-survival.readthedocs.io/en/stable/user_guide/coxnet.html

Cox’s proportional hazard’s model is often an appealing model, because its coefficients can be interpreted in terms of hazard ratio, which often provides valuable insight. However, if we want to estimate the coefficients of many features, the standard Cox model falls apart, because internally it tries to invert a matrix that becomes non-singular due to correlations among features (as we saw above).

4.6.1 Ridge

from sksurv.linear_model import CoxPHSurvivalAnalysis, CoxnetSurvivalAnalysis
from sksurv.preprocessing import OneHotEncoder

from sklearn.model_selection import GridSearchCV, KFold
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

%%time
alphas = 10. ** np.linspace(-4, 4, 50)
coefficients = {}
cph = CoxPHSurvivalAnalysis()
for alpha in alphas:
    #if (i + 1)%10 == 0: print("\rEpisode {}/{}".format(i + 1, n_episodes), end=""); sys.stdout.flush()
    print("\ralpha {}/{}".format(alpha), end=""); sys.stdout.flush()
    cph.set_params(alpha=alpha)
    cph.fit(X_trn_prp, y_trn)
    key = round(alpha, 5)
    coefficients[key] = cph.coef_

coefficients = (pd.DataFrame
    .from_dict(coefficients)
    .rename_axis(index="feature", columns="alpha")
    #-.set_index(get_feature_names(preprocessor)) #. do below
    .set_index(X_trn_prp_df.columns)
               )

# 
# coefficients.head()

def plot_coefficients(coefs, n_highlight):
    _, ax = plt.subplots(figsize=(9, 6))
    n_features = coefs.shape[0]
    alphas = coefs.columns
    for row in coefs.itertuples():
        ax.semilogx(alphas, row[1:], ".-", label=row.Index)

    alpha_min = alphas.min()
    top_coefs = coefs.loc[:, alpha_min].map(abs).sort_values().tail(n_highlight)
    for name in top_coefs.index:
        coef = coefs.loc[name, alpha_min]
        plt.text(
            alpha_min, coef, name + "   ",
            horizontalalignment="right",
            verticalalignment="center"
        )

    ax.yaxis.set_label_position("right")
    ax.yaxis.tick_right()
    ax.grid(True)
    ax.set_xlabel("alpha")
    ax.set_ylabel("coefficient")

# 
# plot_coefficients(coefficients, n_highlight=5)

4.6.2 LASSO

cox_lasso = CoxnetSurvivalAnalysis(l1_ratio=1.0, alpha_min_ratio=0.01)
cox_lasso.fit(X_trn_prp, y_trn)

CoxnetSurvivalAnalysis(alpha_min_ratio=0.01, l1_ratio=1.0)

coefficients_lasso = pd.DataFrame(
    cox_lasso.coef_,
    index=X_trn_prp_df.columns,
    columns=np.round(cox_lasso.alphas_, 5)
)

plot_coefficients(coefficients_lasso, n_highlight=5)

4.6.3 Elastic Net

cox_elastic_net = CoxnetSurvivalAnalysis(l1_ratio=0.9, alpha_min_ratio=0.01)
cox_elastic_net.fit(X_trn_prp, y_trn)

CoxnetSurvivalAnalysis(alpha_min_ratio=0.01, l1_ratio=0.9)

coefficients_elastic_net = pd.DataFrame(
    cox_elastic_net.coef_,
    index=X_trn_prp_df.columns,
    columns=np.round(cox_elastic_net.alphas_, 5)
)

plot_coefficients(coefficients_elastic_net, n_highlight=5)

4.6.4 Choosing penalty strength 𝛼

import warnings
from sklearn.exceptions import FitFailedWarning
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

coxnet_pipe = make_pipeline(
    StandardScaler(),
    CoxnetSurvivalAnalysis(l1_ratio=0.9, alpha_min_ratio=0.01, max_iter=100)
)

Pipeline(
    steps=[
        ('standardscaler', StandardScaler()),
        ('coxnetsurvivalanalysis', CoxnetSurvivalAnalysis(alpha_min_ratio=0.01, l1_ratio=0.9, max_iter=100))
    ])

Pipeline(steps=[('standardscaler', StandardScaler()),
                ('coxnetsurvivalanalysis',
                 CoxnetSurvivalAnalysis(alpha_min_ratio=0.01, l1_ratio=0.9,
                                        max_iter=100))])

%%time
# alphas = 10. ** np.linspace(-4, 4, 50)
# param_grid = {'select__k': np.arange(1, X_trn.shape[1] + 1)}

# estimated_alphas = coxnet_pipe.named_steps["coxnetsurvivalanalysis"].alphas_ #.
cv = KFold(n_splits=5, shuffle=True, random_state=0)
gcv = GridSearchCV(
    make_pipeline(StandardScaler(), CoxnetSurvivalAnalysis(l1_ratio=0.9)),
    #param_grid={"coxnetsurvivalanalysis__alphas": [[v] for v in estimated_alphas]},
    param_grid={"coxnetsurvivalanalysis__alphas": [[v] for v in alphas]}, #.
    cv=cv,
    error_score=0.5,
    n_jobs=4).fit(X_trn_prp, y_trn)

cv_results = pd.DataFrame(gcv.cv_results_)

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/pipeline.py:394: UserWarning: all coefficients are zero, consider decreasing alpha.
  self._final_estimator.fit(Xt, y, **fit_params_last_step)

CPU times: user 450 ms, sys: 44.5 ms, total: 495 ms
Wall time: 18.7 s

cv_results

	mean_fit_time	std_fit_time	mean_score_time	std_score_time	param_coxnetsurvivalanalysis__alphas	params	split0_test_score	split1_test_score	split2_test_score	split3_test_score	split4_test_score	mean_test_score	std_test_score	rank_test_score
0	0.641567	0.151075	0.038515	0.010380	[0.0001]	{'coxnetsurvivalanalysis__alphas': [0.0001]}	0.931531	0.938029	0.931692	0.927769	0.919428	0.929690	0.006098	11
1	0.742988	0.082180	0.036479	0.006518	[0.00014563484775012445]	{'coxnetsurvivalanalysis__alphas': [0.00014563...	0.931575	0.938024	0.931784	0.927774	0.919392	0.929710	0.006117	8
2	0.658041	0.107478	0.041565	0.012953	[0.00021209508879201905]	{'coxnetsurvivalanalysis__alphas': [0.00021209...	0.931557	0.938024	0.931884	0.927802	0.919392	0.929732	0.006121	5
3	0.527088	0.068403	0.041539	0.011195	[0.00030888435964774815]	{'coxnetsurvivalanalysis__alphas': [0.00030888...	0.931592	0.938024	0.931879	0.927835	0.919396	0.929745	0.006120	3
4	0.442916	0.064643	0.041714	0.009012	[0.0004498432668969444]	{'coxnetsurvivalanalysis__alphas': [0.00044984...	0.931597	0.937980	0.931916	0.927826	0.919392	0.929742	0.006113	4
5	0.422237	0.115340	0.040505	0.011708	[0.0006551285568595509]	{'coxnetsurvivalanalysis__alphas': [0.00065512...	0.931610	0.938077	0.931920	0.927821	0.919352	0.929756	0.006154	1
6	0.535060	0.187295	0.038488	0.017137	[0.0009540954763499944]	{'coxnetsurvivalanalysis__alphas': [0.00095409...	0.931680	0.937984	0.931934	0.927906	0.919255	0.929752	0.006162	2
7	0.577488	0.205382	0.039875	0.013807	[0.0013894954943731374]	{'coxnetsurvivalanalysis__alphas': [0.00138949...	0.931685	0.937802	0.931879	0.927967	0.919187	0.929704	0.006129	9
8	0.732059	0.195651	0.036696	0.005739	[0.0020235896477251557]	{'coxnetsurvivalanalysis__alphas': [0.00202358...	0.931738	0.937824	0.931988	0.927859	0.919191	0.929720	0.006152	7
9	0.758118	0.082972	0.039191	0.011675	[0.0029470517025518097]	{'coxnetsurvivalanalysis__alphas': [0.00294705...	0.931799	0.937797	0.932043	0.927911	0.919099	0.929730	0.006181	6
10	0.529540	0.084930	0.037585	0.008908	[0.004291934260128779]	{'coxnetsurvivalanalysis__alphas': [0.00429193...	0.931791	0.937704	0.932070	0.927939	0.918998	0.929700	0.006192	10
11	0.506629	0.087105	0.039250	0.009198	[0.0062505519252739694]	{'coxnetsurvivalanalysis__alphas': [0.00625055...	0.931910	0.937557	0.932107	0.927769	0.918842	0.929637	0.006229	12
12	0.395671	0.039188	0.040258	0.007786	[0.009102981779915217]	{'coxnetsurvivalanalysis__alphas': [0.00910298...	0.931746	0.937388	0.931888	0.927519	0.918797	0.929468	0.006188	13
13	0.332848	0.018462	0.050221	0.006852	[0.013257113655901081]	{'coxnetsurvivalanalysis__alphas': [0.01325711...	0.931650	0.937526	0.931456	0.927368	0.918364	0.929273	0.006343	14
14	0.290543	0.032926	0.042460	0.011817	[0.019306977288832496]	{'coxnetsurvivalanalysis__alphas': [0.01930697...	0.930689	0.937174	0.930126	0.926934	0.917509	0.928486	0.006418	15
15	0.193688	0.017007	0.042062	0.006036	[0.02811768697974228]	{'coxnetsurvivalanalysis__alphas': [0.02811768...	0.928798	0.935603	0.928288	0.925556	0.916416	0.926932	0.006213	16
16	0.130817	0.012389	0.050845	0.008583	[0.040949150623804234]	{'coxnetsurvivalanalysis__alphas': [0.04094915...	0.926515	0.934001	0.926103	0.923093	0.914369	0.924816	0.006341	17
17	0.077204	0.006251	0.060156	0.010282	[0.05963623316594643]	{'coxnetsurvivalanalysis__alphas': [0.05963623...	0.924193	0.931097	0.924065	0.919514	0.912030	0.922179	0.006281	18
18	0.049934	0.004516	0.035734	0.010173	[0.08685113737513521]	{'coxnetsurvivalanalysis__alphas': [0.08685113...	0.920645	0.929292	0.921679	0.917022	0.911227	0.919973	0.005923	19
19	0.034751	0.004112	0.044153	0.006461	[0.12648552168552957]	{'coxnetsurvivalanalysis__alphas': [0.12648552...	0.917406	0.927783	0.917958	0.917079	0.915322	0.919110	0.004426	20
20	0.031743	0.004361	0.035013	0.005995	[0.18420699693267145]	{'coxnetsurvivalanalysis__alphas': [0.18420699...	0.898162	0.922251	0.923461	0.907325	0.925413	0.915322	0.010719	21
21	0.022323	0.010033	0.041810	0.008036	[0.2682695795279725]	{'coxnetsurvivalanalysis__alphas': [0.26826957...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
22	0.020906	0.012749	0.062190	0.008068	[0.3906939937054613]	{'coxnetsurvivalanalysis__alphas': [0.39069399...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
23	0.037980	0.018511	0.068467	0.023919	[0.5689866029018293]	{'coxnetsurvivalanalysis__alphas': [0.56898660...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
24	0.039196	0.011592	0.045779	0.009869	[0.8286427728546842]	{'coxnetsurvivalanalysis__alphas': [0.82864277...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
25	0.031547	0.015219	0.036745	0.014524	[1.2067926406393288]	{'coxnetsurvivalanalysis__alphas': [1.20679264...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
26	0.028867	0.014206	0.035466	0.014775	[1.7575106248547894]	{'coxnetsurvivalanalysis__alphas': [1.75751062...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
27	0.031767	0.009190	0.036941	0.007198	[2.559547922699533]	{'coxnetsurvivalanalysis__alphas': [2.55954792...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
28	0.023922	0.010870	0.043743	0.010469	[3.727593720314938]	{'coxnetsurvivalanalysis__alphas': [3.72759372...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
29	0.025241	0.006658	0.046468	0.007421	[5.428675439323859]	{'coxnetsurvivalanalysis__alphas': [5.42867543...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
30	0.033647	0.007502	0.049515	0.002899	[7.9060432109076855]	{'coxnetsurvivalanalysis__alphas': [7.90604321...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
31	0.035184	0.012833	0.050146	0.005010	[11.513953993264458]	{'coxnetsurvivalanalysis__alphas': [11.5139539...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
32	0.032537	0.005582	0.036215	0.009549	[16.768329368110066]	{'coxnetsurvivalanalysis__alphas': [16.7683293...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
33	0.033725	0.009830	0.027398	0.014742	[24.420530945486497]	{'coxnetsurvivalanalysis__alphas': [24.4205309...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
34	0.025281	0.015240	0.035932	0.012325	[35.564803062231285]	{'coxnetsurvivalanalysis__alphas': [35.5648030...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
35	0.029458	0.006682	0.031290	0.007656	[51.79474679231202]	{'coxnetsurvivalanalysis__alphas': [51.7947467...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
36	0.019044	0.011708	0.041875	0.007795	[75.43120063354607]	{'coxnetsurvivalanalysis__alphas': [75.4312006...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
37	0.029140	0.013129	0.030300	0.007528	[109.85411419875572]	{'coxnetsurvivalanalysis__alphas': [109.854114...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
38	0.021493	0.011760	0.033508	0.008195	[159.98587196060572]	{'coxnetsurvivalanalysis__alphas': [159.985871...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
39	0.028826	0.012988	0.024397	0.012675	[232.99518105153672]	{'coxnetsurvivalanalysis__alphas': [232.995181...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
40	0.028509	0.009678	0.030298	0.007452	[339.3221771895323]	{'coxnetsurvivalanalysis__alphas': [339.322177...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
41	0.027345	0.005549	0.033925	0.010371	[494.1713361323828]	{'coxnetsurvivalanalysis__alphas': [494.171336...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
42	0.030184	0.008610	0.027546	0.010204	[719.6856730011514]	{'coxnetsurvivalanalysis__alphas': [719.685673...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
43	0.022844	0.013882	0.046727	0.011267	[1048.1131341546852]	{'coxnetsurvivalanalysis__alphas': [1048.11313...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
44	0.021944	0.011108	0.046640	0.005771	[1526.4179671752302]	{'coxnetsurvivalanalysis__alphas': [1526.41796...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
45	0.022639	0.009543	0.034386	0.021172	[2222.996482526191]	{'coxnetsurvivalanalysis__alphas': [2222.99648...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
46	0.023126	0.008823	0.042439	0.006511	[3237.45754281764]	{'coxnetsurvivalanalysis__alphas': [3237.45754...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
47	0.028319	0.010468	0.038444	0.006472	[4714.8663634573895]	{'coxnetsurvivalanalysis__alphas': [4714.86636...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
48	0.018925	0.008056	0.039437	0.006805	[6866.488450042998]	{'coxnetsurvivalanalysis__alphas': [6866.48845...	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22
49	0.037058	0.010896	0.021912	0.007598	[10000.0]	{'coxnetsurvivalanalysis__alphas': [10000.0]}	0.500000	0.500000	0.500000	0.500000	0.500000	0.500000	0.000000	22

alphas = cv_results.param_coxnetsurvivalanalysis__alphas.map(lambda x: x[0])
mean = cv_results.mean_test_score
std = cv_results.std_test_score

fig, ax = plt.subplots(figsize=(9, 6))
ax.plot(alphas, mean)
ax.fill_between(alphas, mean - std, mean + std, alpha=.15)
ax.set_xscale("log")
ax.set_ylabel("concordance index")
ax.set_xlabel("alpha")
ax.axvline(gcv.best_params_["coxnetsurvivalanalysis__alphas"][0], c="C1")
ax.axhline(0.5, color="grey", linestyle="--")
ax.grid(True)

best_model = gcv.best_estimator_.named_steps["coxnetsurvivalanalysis"]
best_coefs = pd.DataFrame(
    best_model.coef_,
    index=X_trn_prp_df.columns,
    columns=["coefficient"]
)

non_zero = np.sum(best_coefs.iloc[:, 0] != 0)
print("Number of non-zero coefficients: {}".format(non_zero))

non_zero_coefs = best_coefs.query("coefficient != 0")
coef_order = non_zero_coefs.abs().sort_values("coefficient").index

_, ax = plt.subplots(figsize=(6, 8))
non_zero_coefs.loc[coef_order].plot.barh(ax=ax, legend=False)
ax.set_xlabel("coefficient")
ax.grid(True)

Number of non-zero coefficients: 27

4.6.5 Survival and Cumulative Hazard Function

coxnet_pred = make_pipeline(
    StandardScaler(),
    CoxnetSurvivalAnalysis(l1_ratio=0.9, fit_baseline_model=True)
)
coxnet_pred.set_params(**gcv.best_params_)
coxnet_pred.fit(X_trn_prp, y_trn)

Pipeline(steps=[('standardscaler', StandardScaler()),
                ('coxnetsurvivalanalysis',
                 CoxnetSurvivalAnalysis(alphas=[0.0006551285568595509],
                                        fit_baseline_model=True,
                                        l1_ratio=0.9))])

surv_fns = coxnet_pred.predict_survival_function(X_trn_prp)

#time_points = np.quantile(y["t.tdm"], np.linspace(0, 0.6, 100))
time_points = np.quantile(y_trn['tenure'], np.linspace(0, 0.6, 100)) #.
legend_handles = []
legend_labels = []
_, ax = plt.subplots(figsize=(9, 6))
#for fn, label in zip(surv_fns, X_trn.loc[:, "one-hot__x6_Fiber optic=1"].astype(int)):
for fn, label in zip(surv_fns, X_trn_prp[:, 7].astype(int)): #.
    line, = ax.step(time_points, fn(time_points), where="post",
                   color="C{:d}".format(label), alpha=0.5)
    if len(legend_handles) <= label:
        name = "1" if label == 1 else "0"
        legend_labels.append(name)
        legend_handles.append(line)

ax.legend(legend_handles, legend_labels)
ax.set_xlabel("time")
ax.set_ylabel("Survival probability")
ax.grid(True)

4.7 Random Survival Forests

https://scikit-survival.readthedocs.io/en/stable/user_guide/random-survival-forest.html

4.7.1 Training

%%time
from sksurv.ensemble import RandomSurvivalForest
random_state = 20
rsf = RandomSurvivalForest(
    n_estimators=100, #def 100,
    max_depth=None, #def
    min_samples_split=6, #def 6,
    min_samples_leaf=3, #def 3,
    min_weight_fraction_leaf=0.0, #def
    max_features='sqrt', #def 'auto',
    max_leaf_nodes=None, #def
    bootstrap=True, #def
    oob_score=False, #def
    n_jobs=-1, #def None,
    random_state=random_state, #def None,
    verbose=1, #def 0
    warm_start=False, #def
    max_samples=None #def
)
rsf.fit(X_trn_prp, y_trn)

[Parallel(n_jobs=-1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:    0.2s remaining:    0.0s
[Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed:  1.2min finished

building tree 1 of 100
building tree 2 of 100
building tree 3 of 100
building tree 4 of 100
building tree 5 of 100
building tree 6 of 100
building tree 7 of 100
building tree 8 of 100
building tree 9 of 100
building tree 10 of 100
building tree 11 of 100
building tree 12 of 100
building tree 13 of 100
building tree 14 of 100
building tree 15 of 100
building tree 16 of 100
building tree 17 of 100
building tree 18 of 100
building tree 19 of 100
building tree 20 of 100
building tree 21 of 100
building tree 22 of 100
building tree 23 of 100
building tree 24 of 100
building tree 25 of 100
building tree 26 of 100
building tree 27 of 100
building tree 28 of 100
building tree 29 of 100
building tree 30 of 100
building tree 31 of 100
building tree 32 of 100
building tree 33 of 100
building tree 34 of 100
building tree 35 of 100
building tree 36 of 100
building tree 37 of 100
building tree 38 of 100
building tree 39 of 100
building tree 40 of 100
building tree 41 of 100
building tree 42 of 100
building tree 43 of 100
building tree 44 of 100
building tree 45 of 100
building tree 46 of 100
building tree 47 of 100
building tree 48 of 100
building tree 49 of 100
building tree 50 of 100
building tree 51 of 100
building tree 52 of 100
building tree 53 of 100
building tree 54 of 100
building tree 55 of 100
building tree 56 of 100
building tree 57 of 100
building tree 58 of 100
building tree 59 of 100
building tree 60 of 100
building tree 61 of 100
building tree 62 of 100
building tree 63 of 100
building tree 64 of 100
building tree 65 of 100
building tree 66 of 100
building tree 67 of 100
building tree 68 of 100
building tree 69 of 100
building tree 70 of 100
building tree 71 of 100
building tree 72 of 100
building tree 73 of 100
building tree 74 of 100
building tree 75 of 100
building tree 76 of 100
building tree 77 of 100
building tree 78 of 100
building tree 79 of 100
building tree 80 of 100
building tree 81 of 100
building tree 82 of 100
building tree 83 of 100
building tree 84 of 100
building tree 85 of 100
building tree 86 of 100
building tree 87 of 100
building tree 88 of 100
building tree 89 of 100
building tree 90 of 100
building tree 91 of 100
building tree 92 of 100
building tree 93 of 100
building tree 94 of 100
building tree 95 of 100
building tree 96 of 100
building tree 97 of 100
building tree 98 of 100
building tree 99 of 100
building tree 100 of 100
CPU times: user 1min 11s, sys: 112 ms, total: 1min 11s
Wall time: 1min 13s

RandomSurvivalForest(max_features='sqrt', n_jobs=-1, random_state=20, verbose=2)

rsf.score(X_tst_prp, y_tst) #CI (Concordance Index) score

[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    0.1s finished

0.8957938863762268

The c-index gives information for the whole model. To examine how well the model performed at various time points, we use the cumulative_dynamic_auc metric and visualize it using the code below.

# cumulative_dynamic_auc #8
from sksurv.metrics import (concordance_index_censored,
                            cumulative_dynamic_auc)
times = np.arange(1.1, 13, 1.1)
rsf_chf_funcs = rsf.predict_cumulative_hazard_function(
    X_tst_prp, return_array=False)
rsf_risk_scores = np.row_stack([chf(times) for chf in rsf_chf_funcs])
rsf_auc, rsf_mean_auc = cumulative_dynamic_auc(
    y_trn, y_tst, rsf_risk_scores, times)

#Plotting cumulative dynamic auc
fig, ax = plt.subplots()
plt.plot(times, rsf_auc, "o-", label="RSF (mean AUC = {:.3f})".format(rsf_mean_auc))
plt.xlabel("years from transplant")
plt.ylabel("time-dependent AUC")
plt.legend(loc="lower center")
plt.grid(False)
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)

[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    0.1s finished

ValueError: censoring survival function is zero at one or more time points

4.7.2 Predicting

X_tst_prp_df_sorted = X_tst_prp_df.sort_values(by=["std-scaler__TotalCharges", "std-scaler__MonthlyCharges"])
X_tst_prp_df_sel = pd.concat((X_tst_prp_df_sorted.head(3), X_tst_prp_df_sorted.tail(3)))
X_tst_prp_df_sel

KeyError: ignored

pd.Series(rsf.predict(X_tst_prp_df_sel))

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but RandomSurvivalForest was fitted without feature names
  warnings.warn(

0    25.798703
1    20.841683
2    40.334719
3     0.330962
4     0.340604
5     3.176150
dtype: float64

surv = rsf.predict_survival_function(X_tst_prp_df_sel, return_array=True)
for i, s in enumerate(surv):
    plt.step(rsf.event_times_, s, where="post", label=str(i))
plt.ylabel("Survival probability")
plt.xlabel("Time in days")
plt.legend()
plt.grid(True)

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but RandomSurvivalForest was fitted without feature names
  warnings.warn(

surv = rsf.predict_cumulative_hazard_function(X_tst_prp_df_sel, return_array=True)
for i, s in enumerate(surv):
    plt.step(rsf.event_times_, s, where="post", label=str(i))
plt.ylabel("Cumulative hazard")
plt.xlabel("Time in days")
plt.legend()
plt.grid(True)

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but RandomSurvivalForest was fitted without feature names
  warnings.warn(

4.7.3 Permutation-based Feature Importance

%%time
import eli5
from eli5.sklearn import PermutationImportance

perm = PermutationImportance(rsf, n_iter=15, random_state=random_state)
perm.fit(X_tst_prp, y_tst)
#eli5.show_weights(perm, feature_names=Xt.columns.tolist())
eli5.show_weights(perm, feature_names=X_tst_prp_df.columns.tolist()) #.

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/xgboost/compat.py:36: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  from pandas import MultiIndex, Int64Index

Weight	Feature
0.1645 ± 0.0123	std-scaler__TotalCharges
0.0258 ± 0.0042	one-hot__x13_Two year
0.0083 ± 0.0031	one-hot__x13_One year
0.0019 ± 0.0009	one-hot__x2_Yes
0.0010 ± 0.0009	one-hot__x8_Yes
0.0004 ± 0.0006	one-hot__x5_Yes
0.0001 ± 0.0003	one-hot__x5_No phone service
0.0000 ± 0.0003	one-hot__x4_Yes
-0.0000 ± 0.0004	one-hot__x11_Yes
-0.0001 ± 0.0001	one-hot__x1_1
-0.0001 ± 0.0006	one-hot__x12_Yes
-0.0003 ± 0.0003	one-hot__x15_Mailed check
-0.0003 ± 0.0009	one-hot__x3_Yes
-0.0005 ± 0.0005	one-hot__x0_Male
-0.0006 ± 0.0009	one-hot__x15_Credit card (automatic)
-0.0007 ± 0.0012	one-hot__x14_Yes
-0.0007 ± 0.0008	one-hot__x9_Yes
-0.0013 ± 0.0017	one-hot__x10_Yes
-0.0015 ± 0.0038	one-hot__x6_Fiber optic
-0.0018 ± 0.0026	one-hot__x7_Yes
… 9 more …

4.8 Gradient Boosting Survival Analysis

%%time
from sksurv.ensemble import GradientBoostingSurvivalAnalysis
gbc = GradientBoostingSurvivalAnalysis(
    loss='coxph', #def coxph
    learning_rate=0.1, #def 0.1
    n_estimators=100, #def 100
    criterion='friedman_mse', #def friedman_mse
    min_samples_split=2, #def 2
    min_samples_leaf=1, #def 1
    min_weight_fraction_leaf=0.0, #def 0.0
    max_depth=3, #def 3
    min_impurity_decrease=0.0, #def 0.0
    random_state=0, #def None
    max_features=None, #def None
    max_leaf_nodes=None, #def None
    subsample=1.0, #def 1.0
    dropout_rate=0.0, #def 0.0
    verbose=0, #def 0
    ccp_alpha=0.0, #def 0.0
)
gbc.fit(X_trn_prp, y_trn)
ci_gbc = concordance_index_censored(y_tst["Churn"], y_tst["tenure"], gbc.predict(X_tst_prp))
print(f'The c-index of Gradient Boosting Cox PH is given by {ci_gbc[0]:.3f}')

The c-index of Gradient Boosting Cox PH is given by 0.958
CPU times: user 49.5 s, sys: 26.2 ms, total: 49.6 s
Wall time: 49.6 s

# cumulative_dynamic_auc #8
from sksurv.metrics import (concordance_index_censored,
                            cumulative_dynamic_auc)
times = np.arange(1.1, 13, 1.1)
gbc_chf_funcs = gbc.predict_cumulative_hazard_function(
    X_tst_prp, 
    #return_array=False
)
gbc_risk_scores = np.row_stack([chf(times) for chf in gbc_chf_funcs])
gbc_auc, gbc_mean_auc = cumulative_dynamic_auc(
    y_trn, y_tst, gbc_risk_scores, times)

#Plotting cumulative dynamic auc
fig, ax = plt.subplots()
plt.plot(times, gbc_auc, "o-", label="GBC (mean AUC = {:.3f})".format(gbc_mean_auc))
plt.xlabel("years from transplant")
plt.ylabel("time-dependent AUC")
plt.legend(loc="lower center")
plt.grid(False)
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)

ValueError: censoring survival function is zero at one or more time points

Let’s see how the test performance changes with the ensemble size (n_estimators).

scores_cph_tree = {}

est_cph_tree = GradientBoostingSurvivalAnalysis(
    learning_rate=1.0, max_depth=1, random_state=0
)
for i in range(1, 31):
    n_estimators = i * 5
    est_cph_tree.set_params(n_estimators=n_estimators)
    est_cph_tree.fit(X_trn_prp, y_trn)
    scores_cph_tree[n_estimators] = est_cph_tree.score(X_tst_prp, y_tst)

x, y = zip(*scores_cph_tree.items())
plt.plot(x, y)
plt.xlabel("n_estimators")
plt.ylabel("concordance index")
plt.grid(True)

Let’s repeat the analysis using component-wise least squares base learners.

%%time
from sksurv.ensemble import ComponentwiseGradientBoostingSurvivalAnalysis
scores_cph_ls = {}

est_cph_ls = ComponentwiseGradientBoostingSurvivalAnalysis(
    learning_rate=1.0, random_state=0
)
for i in range(1, 31):
    print('i=', i)
    n_estimators = i*10
    est_cph_ls.set_params(n_estimators=n_estimators)
    est_cph_ls.fit(X_trn_prp, y_trn)
    scores_cph_ls[n_estimators] = est_cph_ls.score(X_tst_prp, y_tst)

x, y = zip(*scores_cph_ls.items())
plt.plot(x, y)
plt.xlabel("n_estimators")
plt.ylabel("concordance index")
plt.grid(True)

The coefficients of the model can be retrieved as follows:

#coef = pd.Series(est_cph_ls.coef_, ["Intercept"] + Xt.columns.tolist())
coef = pd.Series(est_cph_ls.coef_, ["Intercept"] + X_trn_prp_df.columns.tolist()) #.

print("Number of non-zero coefficients:", (coef != 0).sum())
coef_nz = coef[coef != 0]
coef_order = coef_nz.abs().sort_values(ascending=False).index
coef_nz.loc[coef_order]

Number of non-zero coefficients: 18

std-scaler__TotalCharges               -3.057114
one-hot__x13_Two year                  -3.050547
one-hot__x6_No                         -1.283872
one-hot__x13_One year                  -1.110405
std-scaler__MonthlyCharges              0.885347
one-hot__x6_Fiber optic                 0.591382
one-hot__x15_Mailed check               0.349976
one-hot__x15_Electronic check           0.307942
one-hot__x7_Yes                        -0.232877
one-hot__x5_No phone service           -0.122172
one-hot__x2_Yes                        -0.120232
one-hot__x8_Yes                        -0.114403
one-hot__x14_Yes                        0.104997
one-hot__x3_Yes                        -0.103779
one-hot__x10_Yes                       -0.085615
one-hot__x11_Yes                        0.041909
one-hot__x15_Credit card (automatic)   -0.028594
one-hot__x12_Yes                        0.023730
dtype: float64

4.8.4 Accelerated Failure Time Model

est_aft_ls = ComponentwiseGradientBoostingSurvivalAnalysis(
    loss="ipcwls", n_estimators=300, learning_rate=1.0, random_state=0
).fit(X_trn_prp, y_trn)
cindex = est_aft_ls.score(X_tst_prp, y_tst)
print(round(cindex, 3))

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sksurv/ensemble/boosting.py:301: RuntimeWarning: divide by zero encountered in log
  time = numpy.log(time)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sksurv/ensemble/boosting.py:74: RuntimeWarning: invalid value encountered in subtract
  error[component] = squared_norm(residuals - l_pred)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sksurv/ensemble/boosting.py:62: RuntimeWarning: invalid value encountered in multiply
  return X[:, self.component] * self.coef_

ValueError: All-NaN slice encountered

4.8.5 Regularization

%%time
n_estimators = [i*5 for i in range(1, 21)]

estimators = {
    "no regularization": GradientBoostingSurvivalAnalysis(
        learning_rate=1.0, max_depth=1, random_state=0
    ),
    "learning rate": GradientBoostingSurvivalAnalysis(
        learning_rate=0.1, max_depth=1, random_state=0
    ),
    "dropout": GradientBoostingSurvivalAnalysis(
        learning_rate=1.0, dropout_rate=0.1, max_depth=1, random_state=0
    ),
    "subsample": GradientBoostingSurvivalAnalysis(
        learning_rate=1.0, subsample=0.5, max_depth=1, random_state=0
    ),
}

scores_reg = {k: [] for k in estimators.keys()}
for n in n_estimators:
    for name, est in estimators.items():
        print('name:', name) #.
        est.set_params(n_estimators=n)
        est.fit(X_trn_prp, y_trn)
        cindex = est.score(X_tst_prp, y_tst)
        scores_reg[name].append(cindex)

scores_reg = pd.DataFrame(scores_reg, index=n_estimators)

CPU times: user 34min 57s, sys: 165 ms, total: 34min 57s
Wall time: 35min 2s

ax = scores_reg.plot(xlabel="n_estimators", ylabel="concordance index")
ax.grid(True)

4.9 Survival Support Vector Machine

https://scikit-survival.readthedocs.io/en/stable/user_guide/survival-svm.html

4.9.1 Linear Survival Support Vector Machine

from sksurv.metrics import concordance_index_censored
from sksurv.svm import FastSurvivalSVM
from sklearn.model_selection import ShuffleSplit, GridSearchCV

Now, we are essentially ready to start training, but before let’s determine what the amount of censoring for this data is and plot the survival/censoring times.

df.shape

(7043, 20)

df.columns

Index(['gender', 'SeniorCitizen', 'Partner', 'Dependents', 'tenure',
       'PhoneService', 'MultipleLines', 'InternetService', 'OnlineSecurity',
       'OnlineBackup', 'DeviceProtection', 'TechSupport', 'StreamingTV',
       'StreamingMovies', 'Contract', 'PaperlessBilling', 'PaymentMethod',
       'MonthlyCharges', 'TotalCharges', 'Churn'],
      dtype='object')

X_df.shape

(7043, 18)

n_censored = y.shape[0] - y["Churn"].sum()
print("%.1f%% of records are censored" % (n_censored / y.shape[0] * 100))

73.5% of records are censored

plt.figure(figsize=(9, 6))
val, bins, patches = plt.hist((y["tenure"][y["Churn"]],
                               y["tenure"][~y["Churn"]]),
                              bins=30, stacked=True)
_ = plt.legend(patches, ["Time of Churn", "Time of Censoring"])

fastSVM = FastSurvivalSVM(
    alpha=1, #def 1
    rank_ratio=1.0, #def 1.0
    fit_intercept=False, #def False 
    max_iter=10, #def 20
    verbose=False, #def False
    tol=1e-5, #def None
    optimizer=None, #def None
    random_state=random_state, #def None
    timeit=False, #def False
)

fastSVM.fit(X_trn_prp, y_trn)
cindex = fastSVM.score(X_tst_prp, y_tst)
print(round(cindex, 3))

0.935

import eli5
from eli5.sklearn import PermutationImportance

perm = PermutationImportance(fastSVM, n_iter=15, random_state=random_state)
perm.fit(X_tst_prp, y_tst)
eli5.show_weights(perm, feature_names=X_tst_prp_df.columns.tolist()) #.

CPU times: user 6.11 s, sys: 16.8 ms, total: 6.13 s
Wall time: 6.73 s

Weight	Feature
0.2784 ± 0.0225	std-scaler__TotalCharges
0.1098 ± 0.0131	one-hot__x13_Two year
0.0182 ± 0.0046	one-hot__x13_One year
0.0115 ± 0.0031	std-scaler__MonthlyCharges
0.0048 ± 0.0013	one-hot__x15_Mailed check
0.0027 ± 0.0014	one-hot__x15_Electronic check
0.0023 ± 0.0019	one-hot__x6_Fiber optic
0.0013 ± 0.0010	one-hot__x7_Yes
0.0012 ± 0.0010	one-hot__x2_Yes
0.0005 ± 0.0003	one-hot__x5_Yes
0.0003 ± 0.0004	one-hot__x8_Yes
0.0001 ± 0.0005	one-hot__x10_Yes
-0.0000 ± 0.0000	one-hot__x1_1
-0.0000 ± 0.0002	one-hot__x15_Credit card (automatic)
-0.0000 ± 0.0008	one-hot__x12_Yes
-0.0001 ± 0.0001	one-hot__x0_Male
-0.0001 ± 0.0002	one-hot__x9_Yes
-0.0002 ± 0.0006	one-hot__x14_Yes
-0.0002 ± 0.0005	one-hot__x3_Yes
-0.0002 ± 0.0006	one-hot__x7_No internet service
… 9 more …

#Grid search

def score_survival_model(model, X, y):
    prediction = model.predict(X)
    result = concordance_index_censored(y['Churn'], y['tenure'], prediction)
    return result[0]

param_grid = {'alpha': 2.**np.arange(-12, 13, 2)}
cv = ShuffleSplit(n_splits=100, test_size=0.5, random_state=0)
# cv = ShuffleSplit(n_splits=10, test_size=0.5, random_state=0)#.
gcv = GridSearchCV(
    fastSVM, param_grid, 
    scoring=score_survival_model,
    n_jobs=-1, 
    refit=False,
    cv=cv,
    verbose=1)

%%time
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
gcv = gcv.fit(X_trn_prp, y_trn)

Fitting 100 folds for each of 13 candidates, totalling 1300 fits

KeyboardInterrupt:

Let’s check what is the best average performance across 100 random train/test splits we got and the corresponding hyper-parameters.

round(gcv.best_score_, 3), gcv.best_params_

(0.943, {'alpha': 256.0})

Finally, we retrieve all 100 test scores for each parameter setting and visualize their distribution by box plots.

import seaborn as sns
def plot_performance(gcv):
    n_splits = gcv.cv.n_splits
    cv_scores = {"alpha": [], "test_score": [], "split": []}
    order = []
    for i, params in enumerate(gcv.cv_results_["params"]):
        name = "%.5f" % params["alpha"]
        order.append(name)
        for j in range(n_splits):
            vs = gcv.cv_results_["split%d_test_score" % j][i]
            cv_scores["alpha"].append(name)
            cv_scores["test_score"].append(vs)
            cv_scores["split"].append(j)
    df_scores = pd.DataFrame.from_dict(cv_scores)
    _, ax = plt.subplots(figsize=(11, 6))
    sns.boxplot(x="alpha", y="test_score", data=df_scores, order=order, ax=ax)
    _, xtext = plt.xticks()
    for t in xtext:
        t.set_rotation("vertical")

plot_performance(gcv)

We can observe that the model seems to be relative robust with respect to the choice for 𝛼 for this dataset. Let’s fit a model using the 𝛼 value that performed best.

%%time
estimator.set_params(**gcv.best_params_)
estimator.fit(X_trn_prp, y_trn)

CPU times: user 3.21 s, sys: 0 ns, total: 3.21 s
Wall time: 3.22 s

FastSurvivalSVM(alpha=256.0, max_iter=1000, optimizer='avltree', random_state=0,
                tol=1e-05)

It is important to remember that only if the ranking objective is used exclusively (𝑟=1), that predictions denote risk scores, i.e. a higher predicted value indicates shorter survival, a lower value longer survival.

pred = estimator.predict(X_trn_prp_df.iloc[:2])
print(np.round(pred, 3))
print(y_trn[:2])

[0.707 1.205]
[(False, 35.) (False, 15.)]

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/base.py:443: UserWarning: X has feature names, but FastSurvivalSVM was fitted without feature names
  warnings.warn(

4.9.1.1 Regression Objective

Let’s fit a model using the regression objective (𝑟=0) and compare its performance to the ranking model from above.

%%time
ref_estimator = FastSurvivalSVM(rank_ratio=0.0, max_iter=1000, tol=1e-5, random_state=0)
ref_estimator.fit(X_trn_prp, y_trn)

cindex = concordance_index_censored(
    y_trn['Churn'],
    y_trn['tenure'],
    -ref_estimator.predict(X_trn_prp),  # flip sign to obtain risk scores
)
print(round(cindex[0], 3))

ValueError: observed time contains values smaller or equal to zero

Note that concordance_index_censored expects risk scores, therefore, we had to flip the sign of predictions. The resulting performance of the regression model is comparable to the of the ranking model above.

pred = ref_estimator.predict(X_trn_prp_df.iloc[:2])
print(np.round(pred, 3))

4.9.2 Kernel Survival Support Vector Machine

from sksurv.kernels import clinical_kernel
from sksurv.svm import FastKernelSurvivalSVM

%%time
# kernel_matrix = clinical_kernel(data_x)
kernel_matrix = clinical_kernel(X_trn_prp_df)

As with the Linear Survival Support Vector Machine above, we are going to determine the optimal 𝛼 value by using GridSearchCV.

%%time
kssvm = FastKernelSurvivalSVM(optimizer="rbtree", kernel="precomputed", random_state=0)
kgcv = GridSearchCV(
    kssvm, param_grid, 
    scoring=score_survival_model,
    n_jobs=-1, 
    refit=False,
    cv=cv,
    verbose=1)

CPU times: user 42 µs, sys: 0 ns, total: 42 µs
Wall time: 45.3 µs

Note that when using a custom kernel, we do not pass the original data (data_x) to the fit function, but the pre-computed, square kernel matrix.

%%time
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
kgcv = kgcv.fit(kernel_matrix, y_trn)

Fitting 100 folds for each of 13 candidates, totalling 1300 fits

/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)
/home/proj/repos_python/ccc-telecom/venv_ccc-telecom/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:680: ConvergenceWarning: Optimization did not converge: Warning: Maximum number of iterations has been exceeded.
  estimator.fit(X_train, y_train, **fit_params)

KeyboardInterrupt: