nvidia-rapids︱cuDF与pandas一样的DataFrame库

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

鬼畜的大象 · GPU ...· 2 月前 ·

爱听歌的松树 · 【AI成長日記】聊聊GPU規格· 1 月前 ·

率性的水煮鱼 · 教育部97年度母語文學獎· 7 月前 ·

爱喝酒的香瓜 · 嘉兴市教育局门户网站教育人事· 1 年前 ·

越狱的鸵鸟 · Linux内核多线程实现方法 —— ...· 1 年前 ·

讲道义的手链 · 知行合一，从行动开始_王阳明· 1 年前 ·

大鼻子的书包 · 港电影教父邹文怀宣布退休 ...· 1 年前 ·

# for CUDA 9.2
conda install -c rapidsai -c nvidia -c numba -c conda-forge \
    cudf=0.10 python=3.6 cudatoolkit=9.2
# or, for CUDA 10.0
conda install -c rapidsai -c nvidia -c numba -c conda-forge \
    cudf=0.10 python=3.6 cudatoolkit=10.0
# or, for CUDA 10.1
conda install -c rapidsai -c nvidia -c numba -c conda-forge \
    cudf=0.10 python=3.6 cudatoolkit=10.1

docker pull rapidsai/rapidsai:cuda10.1-runtime-ubuntu16.04-py3.7
docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 \
    rapidsai/rapidsai:cuda10.1-runtime-ubuntu16.04-py3.7

import cudf
import numpy as np
from datetime import datetime, timedelta
t0 = datetime.strptime('2018-10-07 12:00:00', '%Y-%m-%d %H:%M:%S')
n = 5
df = cudf.DataFrame({
  'id': np.arange(n),
  'datetimes': np.array([(t0+ timedelta(seconds=x)) for x in range(n)])
df

>>> import cudf
>>> df = cudf.DataFrame([
    (5, "cats", "jump", np.nan),
    (2, "dogs", "dig", 7.5),
    (3, "cows", "moo", -2.1, "occasionally"),
0     1     2     3             4
0  5  cats  jump  null          None
1  2  dogs   dig   7.5          None
2  3  cows   moo  -2.1  occasionally

>>> import pandas as pd
>>> import cudf
>>> pdf = pd.DataFrame({'a': [0, 1, 2, 3],'b': [0.1, 0.2, None, 0.3]})
>>> df = cudf.from_pandas(pdf)
0 0 0.1
1 1 0.2
2 2 nan
3 3 0.3

>>> import cudf
>>> gdf = cudf.DataFrame({'a': [1, 2, None], 'b': [3, None, 5]})
>>> gdf.fillna(4).to_pandas()
0  1  3
1  2  4
2  4  5
>>> gdf.fillna({'a': 3, 'b': 4}).to_pandas()
0  1  3
1  2  4
2  3  5

df = cudf.DataFrame({'a': list(range(20)),
 'b': list(range(20)),
 'c': list(range(20))})
df

df.iloc[1]
a    1
b    1
c    1
Name: 1, dtype: int64

import cudf
import numpy as np
from numba import cuda
df = cudf.DataFrame()
df['in1'] = np.arange(1000, dtype=np.float64)
def kernel(in1, out):
    for i, x in enumerate(in1):
        print('tid:', cuda.threadIdx.x, 'bid:', cuda.blockIdx.x,
              'array size:', in1.size, 'block threads:', cuda.blockDim.x)
        out[i] = x * 2.0
outdf = df.apply_rows(kernel,
                      incols=['in1'],
                      outcols=dict(out=np.float64),
                      kwargs=dict())
print(outdf['in1'].sum()*2.0)
print(outdf['out'].sum())
>>> 999000.0
>>> 999000.0

import cudf
import numpy as np
from numba import cuda
df = cudf.DataFrame()
df['in1'] = np.arange(100, dtype=np.float64)
def kernel(in1, out):
    print('tid:', cuda.threadIdx.x, 'bid:', cuda.blockIdx.x,
          'array size:', in1.size, 'block threads:', cuda.blockDim.x)
    for i in range(cuda.threadIdx.x, in1.size, cuda.blockDim.x):
        out[i] = in1[i] * 2.0
outdf = df.apply_chunks(kernel,
                        incols=['in1'],
                        outcols=dict(out=np.float64),
                        kwargs=dict(),
                        chunks=16,
                        tpb=8)
print(outdf['in1'].sum()*2.0)
print(outdf['out'].sum())
>>> 9900.0
>>> 9900.0

from cudf import DataFrame
df = DataFrame()
df['key'] = [0, 0, 1, 1, 2, 2, 2]
df['val'] = [0, 1, 2, 3, 4, 5, 6]
groups = df.groupby(['key'], method='cudf')
# Define a function to apply to each row in a group
def mult(df):
  df['out'] = df['key'] * df['val']
  return df
result = groups.apply(mult)
print(result)

   key  val  out
0    0    0    0

悟乙己

nvidia-rapids︱cuDF与pandas一样的DataFrame库

文章目录

1 cuDF背景与安装

1.1 背景

1.2 安装

2 一些demo

2.1 新建dataframe

2.2 pandas 与 cuDF切换

2.3 选中某行列

2.4 apply_rows和apply_chunks

2.5 groupby