2015-01-31

Verification of Golden cross and Death cross 5 long moving average's up and down ゴールデンクロスとデッドクロスの検証5 長期移動平均の増減

Verification of Golden cross and Death cross5 ゴールデンクロスとデッドクロスの検証5

I looked for a golden cross with a long average goes up in the last 3 days and the nearest death cross with a long average falls down in the last 3 days. To avoid a cross caused by a fluctuation, I watched a long term average's up and down.

ゴールデンクロスの直近3日間で長期移動平均が増加すること、デッドクロスの直近三日間で長期移動平均が減少することを条件にして、隣接する2クロスの終値を比較して利益が出るか調べた。


Source Code

stocks.db is a database file which contains brand, financial and stocks information about each company.

# -*- coding: utf8 -*- 
#!\usr\local\bin\python3

import pandas as pd
import sqlite3
import datetime
import matplotlib
import matplotlib.pyplot as plt
import pickle

conn = sqlite3.connect('stocks.db') # which includes
c = conn.cursor()

def golden_vs_death(short_term=50, long_term=200):
    ccodes = [i[0] for i in c.execute('SELECT ccode FROM brand_data ORDER BY ccode').fetchall()]
    all = 0
    nice = 0
    for ccode in ccodes:
        df = get_df(ccode)
        if len(df)<=short_term:
            continue
        mavg_short = pd.rolling_mean(df['close'], short_term)
        mavg_long = pd.rolling_mean(df['close'], long_term)
        df['mavg_short'] = pd.Series(mavg_short, index=df.index)
        df['mavg_long'] = pd.Series(mavg_long, index=df.index)
        df['fluctuation'] =df['mavg_short']-df['mavg_long']
        df=df[long_term:]
        signal = {}
        # signal stocks date and golden cross (BUY) or death cross (SELL)
        for i,j in enumerate(df['fluctuation']):
            cond1_long = df['mavg_long'].iloc[i]-df['mavg_long'].iloc[i-1]>0
            cond2_long = df['mavg_long'].iloc[i-1]-df['mavg_long'].iloc[i-2]>0
            if i==0:
                continue             
            # also watch long term moving averages go up or down    
            elif df['fluctuation'].iloc[i]>0 and df['fluctuation'].iloc[i-1]<0 data-blogger-escaped-and="" data-blogger-escaped-cond1_long="" data-blogger-escaped-cond2_long:="" data-blogger-escaped-df.index="" data-blogger-escaped-df="" data-blogger-escaped-elif="" data-blogger-escaped-fluctuation="" data-blogger-escaped-i-1="" data-blogger-escaped-i="" data-blogger-escaped-iloc="" data-blogger-escaped-signal="">0 and not cond1_long and not cond2_long:
                signal[df.index[i]]='SELL'
            else:
                continue
        if len(signal)>1:

            kys=sorted(signal)
            for i,j in enumerate(kys):
                prev_day = kys[i-1]
                the_day = j
                if i==0:
                    pass
                elif signal[prev_day]=='SELL' and signal[the_day]=='BUY':
                    all+=1
                    if df['close'].loc[prev_day] > df['close'].loc[the_day]:
                        print(ccode, prev_day.date(), df['close'].loc[prev_day], 'SELL', '->', j.date(), df['close'].loc[the_day],'BUY RIGHT')
                        nice+=1
                    else:
                        print(ccode, prev_day.date(),df['close'].loc[prev_day], 'SELL', '->', j.date(), df['close'].loc[the_day],'BUY')
                elif signal[prev_day]=='BUY' and signal[the_day]=='SELL':
                    all+=1
                    if df['close'].loc[prev_day] < df['close'].loc[the_day]:
                        print(ccode, prev_day.date(), df['close'].loc[prev_day], 'BUY', '->', j.date(), df['close'].loc[the_day],'SELL RIGHT')
                        nice+=1
                    else:
                        print(ccode, prev_day.date(), df['close'].loc[prev_day], 'BUY', '->',j.date(), df['close'].loc[the_day],'SELL')
                else:
                    pass
    print('{0}/{1} = {2}%'.format(nice, all, round(100*nice/all,1)))


ccc

Results

5 days and 25 days moving averages

1301 2014-05-12 256 SELL -> 2014-06-19 260 BUY
1301 2014-06-19 260 BUY -> 2014-07-07 259 SELL
1301 2014-08-01 258 SELL -> 2014-09-11 254 BUY RIGHT
1301 2014-09-11 254 BUY -> 2014-10-10 251 SELL
1301 2014-10-10 251 SELL -> 2014-10-30 257 BUY
1332 2014-09-11 293.0 SELL -> 2014-10-09 313.0 BUY
1333 2014-05-30 1606.0 BUY -> 2014-07-11 1603.0 SELL
1352 2014-05-16 125.0 SELL -> 2014-06-03 131.0 BUY
1352 2014-06-03 131.0 BUY -> 2014-12-10 128.0 SELL
1376 2014-05-30 882.0 SELL -> 2014-09-29 982.0 BUY
1377 2014-03-27 1379.0 BUY -> 2014-05-15 1403.0 SELL RIGHT
1377 2014-07-11 1364.0 SELL -> 2014-07-17 1396.0 BUY
...
8359 2014-03-04 553.0 SELL -> 2014-03-28 585.0 BUY
8359 2014-03-28 585.0 BUY -> 2014-04-14 543.0 SELL
8359 2014-04-14 543.0 SELL -> 2014-07-18 624.0 BUY
8359 2014-07-18 624.0 BUY -> 2014-08-06 616.0 SELL
8359 2014-08-06 616.0 SELL -> 2014-08-18 631.0 BUY
8360 2014-05-26 451.0 BUY -> 2014-12-12 479.0 SELL RIGHT
8361 2014-03-26 278.0 BUY -> 2014-04-14 263.0 SELL
8361 2014-05-12 267.0 SELL -> 2014-05-27 280.0 BUY
8362 2014-03-28 250.0 BUY -> 2014-04-11 235.0 SELL
8362 2014-04-11 235.0 SELL -> 2014-07-30 249.0 BUY
8362 2014-07-30 249.0 BUY -> 2015-01-07 260.0 SELL RIGHT
766/5835 = 13.1%

5 days and 100 days moving averages

1377 2014-08-15 1398.0 BUY -> 2014-09-01 1385.0 SELL
1377 2014-10-08 1375.0 SELL -> 2014-10-15 1457.0 BUY
1379 2014-07-02 2016.0 BUY -> 2014-08-27 1968.0 SELL
1379 2014-10-06 1962.0 SELL -> 2014-10-28 2035.0 BUY
1383 2014-11-19 1533.0 BUY -> 2014-12-15 1444.0 SELL
1413 2014-11-06 1378.0 BUY -> 2014-11-14 1366.0 SELL
1418 2014-06-09 199.0 SELL -> 2014-12-01 267.0 BUY
1514 2014-08-06 117.0 SELL -> 2014-08-21 122.0 BUY
1514 2014-08-21 122.0 BUY -> 2014-12-15 119.0 SELL
...
8350 2014-09-24 211.0 SELL -> 2014-11-06 213.0 BUY
8355 2014-11-05 1171.0 BUY -> 2014-12-16 1055.0 SELL
8362 2014-07-28 248.0 BUY -> 2014-08-07 238.0 SELL
8362 2014-08-07 238.0 SELL -> 2014-08-15 248.0 BUY
14/958 = 1.5%



Discussion

still bad.

2015-01-30

Verification of Golden Cross and Death Cross 4: With watching if long term moving averages rise up or fall down


Verification of Golden Cross and Death Cross 3:  With watching if long term moving averages rise up or fall down

I compared the close prices on a golden cross and a successive death cross. I extracted a golden cross only when long term averages rise up and a death cross only when long term averages fall down.

2015-01-29

[Python] Verification of Golden cross and Death cross 3

Previous article: 美しい研究室の私: Verification of Golden Cross and Dead Cross 2(ゴールデンクロスとデッドクロスの検証2)

Verification of Golden cross and Death cross 3

It is often said to buy at golden cross and sell at death cross. I tested if it is worth to believe or not.

Method

I collected Japanese stock markets data during 2014/1/1 until 2015/1/23. I run below scripts. stocks.db has historical stock markets data ([Python] jsm使って株式市場のデータをスクレイピングしSQLに保存する). 


# -*- coding: utf8 -*- 
#!\usr\local\bin\python3

import pandas as pd
import sqlite3
import datetime
import matplotlib
import matplotlib.pyplot as plt
import pickle

conn = sqlite3.connect('stocks.db')
c = conn.cursor()

def check_trends_test():
    ccodes = [i[0] for i in c.execute('SELECT ccode FROM brand_data ORDER BY ccode').fetchall()]
    index = get_df(1332).index
    new_df=pd.DataFrame()
    for ccode in ccodes:
        df = get_df(ccode)
        if len(df)<201:
            continue
        mvavg50 = pd.rolling_mean(df['close'], 50)
        mvavg200 = pd.rolling_mean(df['close'], 200)
        trend = [1 if i-j>0 else -1 for i, j in zip(mvavg50, mvavg200)]
        new_trend = [0 for i in trend]
        for i in range(len(trend)):
            if trend[i]>0 and trend[i-1]<0:
                new_trend[i] = 'BUY {0}'.format(df.iloc[i]['close'])
            elif trend[i]<0 and trend[i-1]>0:
                new_trend[i] = 'SELL {0}'.format(df.iloc[i]['close'])
            else:
                pass
        try:
            sum(new_trend[200:])
        except TypeError:
            new_df[ccode] = pd.Series(new_trend, index=df.index)
    pickle.dump(new_df[200:], open('test_trend.pickle', 'wb'))    
    print(new_df.shape)
    print(new_df.iloc[200:])

def golden_vs_death(short_term=50, long_term=200):
    ccodes = pickle.load(open('test_trend.pickle', 'rb')).columns
    all = 0
    nice = 0
    for ccode in ccodes:
        df = get_df(ccode)
        if len(df)<=long_term:
            return None
        mavg_short = pd.rolling_mean(df['close'], short_term)
        mavg_long = pd.rolling_mean(df['close'], long_term)
        df['mavg_short'] = pd.Series(mavg_short, index=df.index)
        df['mavg_long'] = pd.Series(mavg_long, index=df.index)
        df['fluctuation'] =df['mavg_short']-df['mavg_long']
        df=df[long_term:]
        signal = {}
        # signal stocks date and golden cross (BUY) or death cross (SELL)
        for i,j in enumerate(df['fluctuation']):
            cond1_long = df['mavg_long'].iloc[i]-df['mavg_long'].iloc[i-1]>0
            cond2_long = df['mavg_long'].iloc[i-1]-df['mavg_long'].iloc[i-2]>0
            if i==0:
                continue
            #elif df['fluctuation'].iloc[i]>0 and df['fluctuation'].iloc[i-1]<0:
            #    signal[df.index[i]]='BUY'
            #elif df['fluctuation'].iloc[i]<0 and df['fluctuation'].iloc[i-1]>0:
            #    signal[df.index[i]]='SELL'
             
            # also watch long term moving averages go up or down    
            elif df['fluctuation'].iloc[i]>0 and df['fluctuation'].iloc[i-1]<0:
                signal[df.index[i]]='BUY'
            elif df['fluctuation'].iloc[i]<0 and df['fluctuation'].iloc[i-1]>0:
                signal[df.index[i]]='SELL'
            else:
                continue
        if len(signal)>1:

            kys=sorted(signal)
            for i,j in enumerate(kys):
                prev_day = kys[i-1]
                the_day = j
                if i==0:
                    pass
                elif signal[prev_day]=='SELL' and signal[the_day]=='BUY':
                    all+=1
                    if df['close'].loc[prev_day] > df['close'].loc[the_day]:
                        print(ccode, prev_day.date(), df['close'].loc[prev_day], 'SELL', '->', j.date(), df['close'].loc[the_day],'BUY RIGHT')
                        nice+=1
                    else:
                        print(ccode, prev_day.date(),df['close'].loc[prev_day], 'SELL', '->', j.date(), df['close'].loc[the_day],'BUY')
                elif signal[prev_day]=='BUY' and signal[the_day]=='SELL':
                    all+=1
                    if df['close'].loc[prev_day] < df['close'].loc[the_day]:
                        print(ccode, prev_day.date(), df['close'].loc[prev_day], 'BUY', '->', j.date(), df['close'].loc[the_day],'SELL RIGHT')
                        nice+=1
                    else:
                        print(ccode, prev_day.date(), df['close'].loc[prev_day], 'BUY', '->',j.date(), df['close'].loc[the_day],'SELL')
                else:
                    pass
    print('{0}/{1} = {2}%'.format(nice, all, round(100*nice/all,1)))# I changed these arguments.


At first, I made a pickle file which has pandas.DataFrame including just historical data with golden/death cross. Then I counted if that cross makes profits.

Result


A short term is 5 days and a long term is 24 days

1301 2014-02-14 263 BUY -> 2014-02-19 267 SELL RIGHT
1301 2014-02-19 267 SELL -> 2014-02-21 271 BUY
1301 2014-02-21 271 BUY -> 2014-03-17 265 SELL
1301 2014-03-17 265 SELL -> 2014-04-23 263 BUY RIGHT
...
8350 2014-07-29 218.0 SELL -> 2014-09-02 212.0 BUY RIGHT
8350 2014-09-02 212.0 BUY -> 2014-10-03 209.0 SELL
8350 2014-10-03 209.0 SELL -> 2014-10-31 213.0 BUY
8350 2014-10-31 213.0 BUY -> 2014-12-12 220.0 SELL RIGHT
8350 2014-12-12 220.0 SELL -> 2015-01-21 220.0 BUY
2464/8625 = 28.6%

A short term is 5 days and a long term is 100 days

1301 2014-07-23 266 BUY -> 2014-07-31 258 SELL
1301 2014-07-31 258 SELL -> 2014-09-29 263 BUY
1301 2014-09-29 263 BUY -> 2014-10-08 256 SELL
1301 2014-10-08 256 SELL -> 2014-10-31 260 BUY
...
8350 2014-09-24 211.0 SELL -> 2014-09-25 213.0 BUY
8350 2014-09-25 213.0 BUY -> 2014-10-02 210.0 SELL
8350 2014-10-02 210.0 SELL -> 2014-11-06 213.0 BUY
176/2581 = 6.8%

A short term is 5 days and a long term is 200 days

1379 2014-10-28 2035.0 BUY -> 2014-12-01 1989.0 SELL
1379 2014-12-01 1989.0 SELL -> 2014-12-03 1991.0 BUY
1379 2014-12-03 1991.0 BUY -> 2014-12-10 1987.0 SELL
1379 2014-12-10 1987.0 SELL -> 2014-12-30 2001.0 BUY
...
8316 2014-11-05 4538.5 BUY -> 2014-12-17 4121.0 SELL
8316 2014-12-17 4121.0 SELL -> 2014-12-24 4358.5 BUY
8316 2014-12-24 4358.5 BUY -> 2015-01-08 4159.5 SELL
23/805 = 2.9%

A short term is 10 days and a long term is 100 days

1301 2014-07-28 260 BUY -> 2014-08-05 255 SELL
1301 2014-08-05 255 SELL -> 2014-09-30 260 BUY
1301 2014-09-30 260 BUY -> 2014-10-10 251 SELL
1301 2014-10-10 251 SELL -> 2014-11-07 261 BUY
...
8350 2014-09-26 212.0 SELL -> 2014-09-30 213.0 BUY
8350 2014-09-30 213.0 BUY -> 2014-10-03 209.0 SELL
8350 2014-10-03 209.0 SELL -> 2014-11-11 222.0 BUY
114/1762 = 6.5%

A short term is 10 days and a long term is 200 days

1379 2014-10-31 2038.0 BUY -> 2014-12-09 1983.0 SELL
1379 2014-12-09 1983.0 SELL -> 2015-01-08 2000.0 BUY
1383 2014-11-20 1551.0 BUY -> 2014-12-18 1452.0 SELL
1413 2014-11-05 1410.0 BUY -> 2014-11-26 1339.0 SELL
...
8289 2014-11-05 994.0 BUY -> 2014-12-03 935.0 SELL
8309 2014-11-11 473.4 BUY -> 2015-01-14 424.0 SELL
8316 2014-11-11 4483.5 BUY -> 2015-01-14 4007.5 SELL
21/472 = 4.4%

A short term is 25 days and a long term is 200 days

1379 2014-11-21 2005.0 BUY -> 2014-12-15 1944.0 SELL
1379 2014-12-15 1944.0 SELL -> 2015-01-21 2075.0 BUY
1383 2014-11-27 1572.0 BUY -> 2014-12-30 1439.0 SELL
...
8217 2014-10-28 870.0 SELL -> 2014-11-25 963.0 BUY
8217 2014-11-25 963.0 BUY -> 2014-12-26 927.0 SELL
8282 2014-12-04 2935.0 SELL -> 2014-12-30 3170.0 BUY
8309 2014-11-26 490.6 BUY -> 2015-01-21 420.5 SELL
8316 2014-11-25 4528.0 BUY -> 2015-01-20 4090.0 SELL
10/175 = 5.7%

Conclusion

The results were terrible. Though most of beginner books say, "You should buy at golden cross and sell at death cross", but you don't get profit. The best ratio you make profit is 29% when a short term is 5 days and a long term is 24 days. Most of time you loose. That's why many people lost their money buy trading.

Discussion

In many cases, there are cross which caused just by market's fluctuation. I want to get rid of them. One of the solutions is to see if long term averages go up or fall down at a cross. I'll consider it next time.

2015-01-27

Verification of Golden Cross and Dead Cross 2(ゴールデンクロスとデッドクロスの検証2)

Verification of Golden Cross and Dead Cross 2(ゴールデンクロスとデッドクロスの検証2)

*** I wrongly used only the last 50 days data. *** まちがえて直近50日間のデータのみ使って解析していました。


It is often said that buying at golden cross and selling at death cross is good. I tested if it is true or not. The data were closing stock prices in Japanese stock market during January 1, 2014 to January 23, 2015.

How often you buy low and sell high when you buy at golden cross and sell at death cross (単にゴールデンクロスで買い、デッドクロスで売ると、差額が正になるのは何割か)

stock cord, date, close, buy/sell -> date, close, buy/sell (, RIGHT)
if you buy low and sell high, flag is "RIGHT".

証券コード、日付、終値、売買 -> 日付、終値、売買 (, RIGHT)
で表した。売買というのは、ゴールデンクロスならBUY, デッドクロスなら売り。2クロスの終値を比較して、安く売って高く買っていた場合、RIGHTというフラグをつけた


5 days and 24 days moving averages (5日平均、24日平均による移動平均線)

1942 2015-01-22 685.0 SELL -> 2015-01-23 710.0 BUY
1948 2015-01-20 214.0 BUY -> 2015-01-22 214.0 SELL
1948 2015-01-22 214.0 SELL -> 2015-01-23 213.0 BUY RIGHT
2498 2015-01-20 692.0 SELL -> 2015-01-23 693.0 BUY
3001 2015-01-20 1261.0 BUY -> 2015-01-22 1239.0 SELL
3408 2015-01-20 170.0 SELL -> 2015-01-21 175.0 BUY
3738 2015-01-21 1229.0 SELL -> 2015-01-23 1254.0 BUY
4471 2015-01-21 851.0 SELL -> 2015-01-22 879.0 BUY
4645 2015-01-20 244.0 SELL -> 2015-01-23 244.0 BUY
6085 2015-01-20 3160.0 SELL -> 2015-01-22 3205.0 BUY
6258 2015-01-20 660.0 SELL -> 2015-01-23 673.0 BUY
6408 2015-01-21 326.0 BUY -> 2015-01-22 321.0 SELL
7768 2015-01-20 486.0 SELL -> 2015-01-23 489.0 BUY
7863 2015-01-20 325.0 BUY -> 2015-01-22 319.0 SELL
8154 2015-01-21 1433.0 BUY -> 2015-01-22 1416.0 SELL
1 out of 15

[Python] jsm使って株式市場のデータをスクレイピングしSQLに保存する

Python jsmを使って日本の株式市場をスクレイピング

日本の株式データをとるには、公開APIが少なく、あっても有料なものが多いので、Yahoo!Financeからスクレイピングするケースが多いらしい。

株式データを2004/1/1から今日まで取得してSQLに格納するコードを書いた。databaseファイルは82MBになった。


#!/usr/local/bin/python3
# -*- coding: utf-8 -*-

import datetime
import jsm
import sys
import sqlite3
import pickle
import os
import urllib

class Stocks():
    def __init__(self):
        self._conn = sqlite3.connect('stocks.db')
        self._cur = self._conn.cursor()
           
    def get_ccodes(self):
        '''
        return a ccodes list
        '''
        try:
            self._cur.execute('SELECT ccode FROM brand_data')
            ccodes = self._cur.fetchall()
            if len(ccodes) == 0:
                print('The table brand_data exists but has no data.')
                self.refresh_brand()
                self._cur.execute('SELECT ccode FROM brand_data') 
                ccodes = self._cur.fetchall()
        except:
            print('The table brand_data does not exist.')
            self.refresh_brand()
            self._cur.execute('SELECT ccode FROM brand_data')
            ccodes = self._cur.fetchall()
        self.ccodes = [i[0] for i in ccodes]
        return self.ccodes

    def refresh_brand(self):
        """
        update sql brand_data and set self.list_of_dict_brand
        update ccodes list
        """
        print('refreshing brand data.')
        sql = '''CREATE TABLE IF NOT EXISTS brand_data(
        ccode int, industry_code int, industry_name text, market 

text, name text, info text, datetime text, PRIMARY KEY(ccode)
        )'''

        self._cur.execute(sql)
        b = jsm.Brand()
        q = jsm.Quotes()
        IDS = b.IDS
        list_of_tuple_brand = []
        print('IDS',IDS)
        for industry_code in IDS.keys():
            try:
                industry_name = IDS[industry_code]
                brand_data = q.get_brand(industry_code)
                [list_of_tuple_brand.append((brand.ccode,
                                industry_code,
                                industry_name,
                                brand.market,
                                brand.name,
                                brand.info,
                                datetime.datetime.now()
                                )) for brand in brand_data]
                print('Industry Code: {0}'.format(industry_code))
            except:
                print('ERROR in brand data: {0}'.format

(industry_code))
        with open('brand.pickle', 'wb') as f:
            pickle.dump(list_of_tuple_brand, f)
        # Replace brand sql
        sql = '''REPLACE INTO brand_data(ccode, industry_code, 

industry_name, market, name, info, datetime) VALUES 

(?,?,?,?,?,?,?)'''
        self._cur.executemany(sql,list_of_tuple_brand)
    
    def refresh_finance(self):
        '''refresh finance information in finance_data'''
        print('refreshing finance data.')
        q = jsm.Quotes()
        sql = '''CREATE TABLE IF NOT EXISTS finance_data(
        ccode int,
        market_cap real,
        shares_issued real,
        dividend_yield real,
        dividend_one real,
        per real,
        pbr real,
        eps real,
        bps real,
        price_min real,
        round_lot real,
        years_high int,
        years_low int,
        datetime text,
        PRIMARY KEY(ccode)
        )'''
        self._cur.execute(sql)
        ccodes = self.get_ccodes()
        list_of_tuple_finance = []
        for ccode in ccodes:
            try:
                finance_data = q.get_finance(ccode)
                list_of_tuple_finance.append((ccode,
                                             finance_data.market_cap,
                                             

finance_data.shares_issued,
                                             

finance_data.dividend_yield,
                                             

finance_data.dividend_one,
                                             finance_data.per,
                                             finance_data.pbr,
                                             finance_data.eps,
                                             finance_data.bps,
                                             finance_data.price_min,
                                             finance_data.round_lot,
                                             finance_data.years_high,
                                             finance_data.years_low,
                                             datetime.datetime.now

()))
                print('ccode: {0}'.format(ccode))
            except:
                print("Error in Financial Data: ", ccode)
        sql = '''REPLACE INTO finance_data
        (ccode, market_cap, shares_issued, dividend_yield, 

dividend_one, per, pbr, eps, bps, price_min, round_lot, years_high, 

years_low, datetime)
        VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)'''
        self._cur.executemany(sql,list_of_tuple_finance)
        
    def refresh_stocks(self, ccode):
        """
        refreshing stocks
        """
        sql = '''CREATE TABLE IF NOT EXISTS stocks{0}
        (date text, volume real, open real, high real, low real, 

close real, PRIMARY KEY(date))'''.format(ccode)
        self._cur.execute(sql)
        q = jsm.Quotes()
        try:
            with open('stocks_ts.pickle','rb') as f:
                stocks_ts = pickle.load(f)
        except:
            stocks_ts = {}

        # set periods to get stocks
        if ccode in stocks_ts.keys():
            start_date = stocks_ts[ccode] + datetime.timedelta

(days=1)
        else:
            start_date = datetime.date(2014,1,1)
        end_date = datetime.date.today()
        if start_date<=end_date:

            try:
                quote_data = q.get_historical_prices

(ccode,jsm.DAILY,start_date=start_date,end_date=end_date)
                sql = '''REPLACE INTO stocks{0}
                (date, volume, open, high, low, close) VALUES (?, ?, 

?, ?, ?, ?)'''.format(ccode)
                values = 

[(q.date,q.volume,q.open,q.high,q.low,q.close) for q in quote_data]
                self._cur.executemany(sql, values)
                # update timestamp
                stocks_ts[ccode] = datetime.datetime.strptime

(self._cur.execute('SELECT date FROM stocks{0} ORDER BY date'.format

(ccode)).fetchall()[-1][0], '%Y-%m-%d %H:%M:%S').date()
                with open('stocks_ts.pickle', 'wb') as f:
                    pickle.dump(stocks_ts, f)
                print('ccode: {0} from {1} to {2}'.format(ccode, 

start_date, end_date))
            except jsm.exceptions.CCODENotFoundException as e:
                print(ccode, ': ', e)              
            except urllib.error.HTTPError as e:
                print('ccode: {0}; HTTP ERROR CODE: {1}'.format

(ccode, e.code)) 
        else:
            print('ccode: {0} is already updated.'.format(ccode))
    def commit(self):
        self._conn.commit()
        
    def close(self):
        self._cur.close()
        self._conn.close()
        
if __name__=='__main__':
    stocks = Stocks()
    stocks.get_ccodes()
    [stocks.refresh_stocks(ccode) for ccode in stocks.ccodes]
    stocks.commit()
    stocks.close()


今後、解析用のメソッドも書いていく


参考
株価をYahoo Financeから取得する[Python]

株価のトレンド変化について検証 -*- ゴールデンクロスとデスクロスについて -+-.

株価のトレンド変化

株価は、短期トレンド、長期トレンドなどがある。短期トレンドが長期トレンドを上回れば、その後は上がるだろうと (golden cross)、また、短期トレンドが長期トレンドを下回れば、その後は下がるだろうと予想できる (death cross)。短期、長期は10日や15日、一年以下など、100日や200日、一年以上など様々。

嫌な例。下がると思って売ったのに高くなって買う。

2015-01-22

日本の証券市場の株価のヒストリカルデータをとる

先週末の金曜日に書き上げた、日本の株式市場の株価・財務データを昨年元日から今日まで取得する書き上げた。今日、デバッグがおよそ終わったらしく、ひたすらAWS上のSQLに格納している。証券コード約4000まで済んだあたりで、Yahoo Financeからtoo many requestsのHTTP ERROR CODE 999 を受けている。


cygwinのスクリーンは写真上。時間をおいてまた続ける。


2015-01-14

jsm

次のコードをまずpython3向けに直した。
株価をYahoo Financeから取得する[Python] - Qiita

import jsm



"""Get Brand"""

b = jsm.Brand()

q = jsm.Quotes()

IDS = b.IDS

list_of_dict_brand = []

for industry_code in IDS.keys():

    print(industry_code)

    industry_name = IDS[industry_code]

    brand_data = q.get_brand(industry_code)

    [list_of_dict_brand.append({"ccode":brand.ccode,

                                "industry_code":industry_code,

                                "industry_name":industry_name,

                                "market":brand.market,

                                "name":brand.name,

                                "info":brand.info

                                }) for brand in brand_data]



""" get finance data """

ccodes = [i["ccode"] for i in list_of_dict_brand]



list_of_dict_finance = []

cnt = 0

for ccode in ccodes: #取得した証券コードでループをまわす。

    print(ccode)

    try:

        finance_data = q.get_finance(ccode)

        list_of_dict_finance.append({"ccode":ccode,

                                    "market_cap":finance_data.market_cap,

                                    "shares_issued":finance_data.shares_issued,

                                    "dividend_yield":finance_data.dividend_yield,

                                    "dividend_one":finance_data.dividend_one,

                                    "per":finance_data.per,

                                    "pbr":finance_data.pbr,

                                    "eps":finance_data.eps,

                                    "bps":finance_data.bps,

                                    "price_min":finance_data.price_min,

                                    "round_lot":finance_data.round_lot})

    except:

        print("Error in Financial Data " , ccode)



""" get stock. This takes a time more than minutes"""

list_of_dict_stock = []

start_date = datetime.date(2014,1,1)

end_date = datetime.date.today()

for ccode in ccodes:

    try:

        quote_data = q.get_historical_prices(ccode,jsm.DAILY,start_date=start_date,end_date=end_date)

        if quote_data is None:

            print('ccode {} is None'.format(ccode))

            continue



        [list_of_dict_stock.append({"ccode":ccode,

                     "date":quote.date,

                     "open":quote.open,

                     "high":quote.high,

                     "low":quote.low,

                     "close":quote.close,

                     "volume":quote.volume}) for quote in quote_data]

        print(ccode)



    except:

        print("Error in Quote Data", ccode)



####
次のような結果が返る

list_of_dict_brand[:4]

[{'name': '特種東海製紙(株)',
  'industry_code': '3150',
  'market': '東証1部',
  'ccode': '3708',
  'industry_name': 'パルプ・紙',
  'info': '特種製紙と東海パルプが07年統合、独立系。特殊紙と段ボール原紙用板紙が両輪。製紙8位級'},
 {'name': '王子ホールディングス(株)',
  'industry_code': '3150',
  'market': '東証1部',
  'ccode': '3861',
  'industry_name': 'パルプ・紙',
  'info': '1873年創立。製紙国内首位、うち洋紙2位・板紙1位。アジアや南米など成長市場へ展開'},
 {'name': '日本製紙(株)',
  'industry_code': '3150',
  'market': '東証1部',
  'ccode': '3863',
  'industry_name': 'パルプ・紙',
  'info': '旧王子製紙のうち十條製紙を継承、01年大昭和製紙統合。製紙国内2位、うち洋紙1位・板紙3位'},
 {'name': '三菱製紙(株)',
  'industry_code': '3150',
  'market': '東証1部',
  'ccode': '3864',
  'industry_name': 'パルプ・紙',
  'info': '業界5位で印刷・情報用紙大手。写真感光材などイメージングとの両輪。北越紀州と販社統合へ'}]

{'ccode': '2388',
 'date': datetime.datetime(2015, 1, 13, 0, 0),
 'volume': 15000,
 'open': 164,
 'high': 165,
 'low': 163,
 'close': 165}


2015-01-09

Change Home Directory of Cygwin

To change Home directory of Cygwin on Windows

You just need create new path.
Click on -> Control Panel -> System -> Advanced -> Environment Variables
Create new path:
    Path variable: HOME
    Path value: /cygdrive/c/users/username

Python3.4 Virtual Environments on Windows (仮想環境)

The creation of virtual environments

Your pyvenv is installed by default in C:\Python34\Tools\Scripts. To create virtual environments, if you have the relevant PATH and PATHEXT settings:

C:>py -3.4 -m venv c:\path\to\wherever\you\want\it

or equvalently

C:>C:\Python34\Python C:\Python34\Tools\Scripts\pyvenv.py C:\path\to\wherever\you\want\it

To activate the virtual environment


C:>c:\path\to\wherever\you\want\it\Scripts\activate.bat

Install libraries on virtual environments

On windows to install numpy, scipy and matplotlib, it is easier to download installers from

 Python Extension Packages for Windows - Christoph Gohlke

than using pip or easy_install. For example, to install numpy-MLK, download its installer and try
(venv) C:\pip C:\Users\Username\Downloads\numpy-MKL-1.9.1.win32-py3.4.exe

Probably you need install Scipy-stack to use scikit-learn.

When python is uninstalled

When you uninstall python, all files and folders in virtual environments still remain. If you try to use python in virtual environment, you will be said that python34.dll is not found and reinstall it.


References

2015-01-05

No easy_install and pip after re-installing Python 3.4

Missing Some Files in Scripts After Re-installing Python 3.4 

No easy_install and pip after re-installing Python

When I removed Python 3.4 and re-installed it, I came across a problem. I did not find easy_install nor pip. This is because, I suppose, uninstallation of Python 3.4 removes easy_install and pip from C:\Python34\Scripts but re-installing Python 3.4 dose not override C:\Python34\Scripts. Even when you uninstall Python 3.4  by selecting "Uninstall Python" from Start Menu on Windows, the folder C:\Python34 still remains. So dose C:\Python34\Scripts and some other ones in it.


Delete C:\Python34\Scripts seems to work finely.

I dared to delete C:\Python34 directly into "Rycycle bin" and installed Python 3.4 again. C:\Python34\Scripts was created. In that, I found these files.

c:\Python34\Scripts>dir  /b
easy_install-3.4.exe
easy_install.exe
pip.exe
pip3.4.exe
pip3.exe

pyvenv is also installed in C:\Python34\Tools\Scripts

飲食店への商品持ち込みを批判する人への反論

持ち込み商品の反対意見について反反論がある。持ち込み商品については、それはマナー違反だからと言う人が多い。商品を持ち込んでいる客がいると、注意したところでバイト代の上がらないバイト、給料の上がらない正社員は注意をし、さらに周りの客でも注意をする人がいる。彼らにその理由を聞くと、「...