Huge file handling and sorting using python -

-

im working on program uses file having data in format - 6 columns , dynamic no. of rows.

the file got testing 26 mb , following program converts first 3 columns 3 different lists.

f = open('foo', 'r') print('running...') = [] b = [] c = [] line in f:     x = (line.split(' '))     a.append(x[0])     b.append(x[1])     c.append(x[2]) print(a,b,c,sep='\n')

i have rechecked program , logic looks correct , when implemented on small file works when use program 26 mb file stops responding.

description of program: program opens file name 'foo' , implements line line of file. splits line parts based on separator defined argument in .split() method. in program have used white space separator in text file data separated using white spaces.

im not able figure out why program stops responding , need it!

if use numpy, can use genfromtxt:

import numpy np  a,b,c=np.genfromtxt('foo',usecols=[0,1,2],unpack=true)

does work large file?

edit:

ok, tried on file, , seems work fine. i'm not sure problem is.

in [1]: numpy import genfromtxt  in [2]: a,b,c=genfromtxt('foo',usecols=[0,1,2],unpack=true)  in [3]: out[3]:  array([ 406.954744,  406.828508,  406.906079, ...,  408.944226,         408.833872,  408.788698])  in [4]: b out[4]:  array([ 261.445358,  261.454366,  261.602131, ...,  260.46189 ,         260.252377,  260.650606])  in [5]: c out[5]:  array([ 17.451789,  17.582017,  17.388673, ...,  26.41099 ,  26.481148,         26.606282])  in [6]: print len(a), len(b), len(c) 419040 419040 419040

Comments

Post a Comment

Popular posts from this blog

angularjs - Redirect to a new template in angular js -

-
i new angularjs, please consider mistakes. the scenario is, have index.html page , login.html page, index , login have different layout different header footer , all. other pages expect login share same layout index.html now, how can navigate login.html using anuglar ui router. myapp.config(function ($stateprovider, $urlrouterprovider) { $stateprovider.state('home', { url: '/', templateurl: 'app/views/site/home.html', controller: 'homectrl' }).state('login', { url: '/login', templateurl: 'app/views/site/login.html' }) }); this not work loads content of login index keeping header , footer same of index. know state supposed work way. now, there way can call new view using ui.router. i appreciate help. thanks you set multiple named views ( https://github.com/angular-ui/ui-router/wiki/multiple-named-views ). this allow make header , footer separate views can ...
Read more

Oracle SQL. How to select specific IDs where value of columns specify criteria? -

-
i have following statement/query: select table1.file_id, table1.status, table2.status table1 inner join table2 on table1.file_id = table2.file_id; the result follows: file id | status table 1 | status table 2 ----------------------------------------------------- 5500002 | ax | ics 5500002 | ax | icso 5500002 | axo | ics 5500003 | ax | ics 5500003 | ax | ics 5500003 | ax | ics 5500004 | null | icso test 5500004 | axo | ics 5500004 | ax | null i need each file id check 4 statuses: - ax - axo - ics - icso i want run select on file id s cover 4 statuses. can see file id occurs multiple times , i'd want query return file id fulfills requirement, in instance return 5500002 , 5500004. for 550000...
Read more

Desktop Launcher for Python Script Starts Program in Wrong Path (Linux) -

-
i can not launch python script .desktop launcher created on linux mint 17.1 cinnamon. the problem script launched in wrong path - namely home folder instead of directory placed in. thereby can not find other vital files accompanying in folder , hence not work. to examine misbehaviour created short script check folder python script executing in: #!/usr/bin/env python import subprocess import time subprocess.call(["pwd"], shell=true) time.sleep(7) # chance read output executing own folder gives output: /home/myusername/pythonprojects i setting desktop launcher via nemo's menu. executing same script yields: /home/myusername i not understand behaviour. how create working desktop launcher python script? the page https://linuxcritic.wordpress.com/2010/04/07/anatomy-of-a-desktop-file/ describes format of .desktop files. you may note "path" element, specifies working directory file run in. in case want desktop file specified path=/...
Read more