Pandas 데이터 프레임에서 NaN 값이 포함 된 열을 찾는 방법 (Python)

Pandas 데이터 프레임에서 NaN 값이 포함 된 열을 찾는 방법 (Python)

---

여기 저기 흩어져있는 가능한 NaN 값을 포함하는 pandas 데이터 프레임이 주어지면 :

질문 : NaN 값이 포함 된 열을 어떻게 확인합니까? 특히 NaN이 포함 된 열 이름 목록을 얻을 수 있습니까?

---

업데이트 : Pandas 0.22.0 사용

최신 Pandas 버전에는 'DataFrame.isna ()' 및 'DataFrame.notna ()'라는 새로운 메서드가 있습니다.

In [71]: df
Out[71]:
     a    b  c
0  NaN  7.0  0
1  0.0  NaN  4
2  2.0  NaN  4
3  1.0  7.0  0
4  1.0  3.0  9
5  7.0  4.0  9
6  2.0  6.0  9
7  9.0  6.0  4
8  3.0  0.0  9
9  9.0  0.0  1

In [72]: df.isna().any()
Out[72]:
a     True
b     True
c    False
dtype: bool

열 목록으로 :

In [74]: df.columns[df.isna().any()].tolist()
Out[74]: ['a', 'b']

해당 열을 선택하려면 (하나 이상의 NaN값 포함) :

In [73]: df.loc[:, df.isna().any()]
Out[73]:
     a    b
0  NaN  7.0
1  0.0  NaN
2  2.0  NaN
3  1.0  7.0
4  1.0  3.0
5  7.0  4.0
6  2.0  6.0
7  9.0  6.0
8  3.0  0.0
9  9.0  0.0

---

이전 답변 :

isnull ()을 사용해보십시오 .

In [97]: df
Out[97]:
     a    b  c
0  NaN  7.0  0
1  0.0  NaN  4
2  2.0  NaN  4
3  1.0  7.0  0
4  1.0  3.0  9
5  7.0  4.0  9
6  2.0  6.0  9
7  9.0  6.0  4
8  3.0  0.0  9
9  9.0  0.0  1

In [98]: pd.isnull(df).sum() > 0
Out[98]:
a     True
b     True
c    False
dtype: bool

또는 @root가 더 명확한 버전을 제안했습니다.

In [5]: df.isnull().any()
Out[5]:
a     True
b     True
c    False
dtype: bool

In [7]: df.columns[df.isnull().any()].tolist()
Out[7]: ['a', 'b']

하위 집합 선택-하나 이상의 NaN값을 포함하는 모든 열 :

In [31]: df.loc[:, df.isnull().any()]
Out[31]:
     a    b
0  NaN  7.0
1  0.0  NaN
2  2.0  NaN
3  1.0  7.0
4  1.0  3.0
5  7.0  4.0
6  2.0  6.0
7  9.0  6.0
8  3.0  0.0
9  9.0  0.0

---

You can use df.isnull().sum(). It shows all columns and the total NaNs of each feature.

---

i use these three lines of code to print out the column names which contain at least one null value:

for column in dataframe:
    if dataframe[column].isnull().any():
       print('{0} has {1} null values'.format(column, dataframe[column].isnull().sum()))

---

I had a problem where I had to many columns to visually inspect on the screen so a short list comp that filters and returns the offending columns is

nan_cols = [i for i in df.columns if df[i].isnull().any()]

if that's helpful to anyone

---

Both of these should work:

df.isnull().sum()
df.isna().sum()

DataFrame methods isna() or isnull() are completely identical.

Note: Empty strings '' is considered as False (not considered NA)

---

In datasets having large number of columns its even better to see how many columns contain null values and how many don't.

print("No. of columns containing null values")
print(len(df.columns[df.isna().any()]))

print("No. of columns not containing null values")
print(len(df.columns[df.notna().all()]))

print("Total no. of columns in the dataframe")
print(len(df.columns))

For example in my dataframe it contained 82 columns, of which 19 contained at least one null value.

Further you can also automatically remove cols and rows depending on which has more null values
Here is the code which does this intelligently:

df.drop(df.columns[df.isna().sum()>len(df.columns)],axis = 1,inplace = True)
df.dropna(axis = 0,inplace = True)

Note: Above code removes all of your null values. If you want null values, process them before.

참고URL : https://stackoverflow.com/questions/36226083/how-to-find-which-columns-contain-any-nan-value-in-pandas-dataframe-python