Question

How to use a while loop over a file to pass a search variable to a python pandas dataframe

I have two files: a Category file and a Spending file. I want to be able to set up a while loop to pass a variable from the Category file to a filter instruction for the Spending record. My spending category file (ecat) has a list of spending categories such as: “operations”, “marketing”, “”, “maintenance”, etc. The Spending file (outgo) has a table-like list of spending items like:

Date:Category:Amount:Remarks (**those are the headings)
     Date     Category     Amount                     Remarks
0   201101:   Operations:     200.00:                   paperwork
1   201201:   Operations:     190.00:                   who knows
2   210109:  Contractors:    1050.00:        Political assessment
3   201001:  Engineering:    2550.00:               new equipment
4   201203:   Operations:  300000.00:               land purchase

My filter command is like this:

df=pd.read_csv('outgo',delimiter=':')

So, the while loop I have set up is as follows:

 with open('ecats', 'r') as catfile:
          cat=catfile.readline()
          while cat:
            print(cat)
            x=df[df.Category.str.contains(cat)]
            print(x)
            cat=catfile.readline()

The problem seems to be that the “cat” variable is not being passed to the filter instruction. It returns an empty dataframe[].

The print command duly prints out which category is being accessed as the while loops through the file but the variable gets lost or blocked somewhere on the way to the filter command x=df[df.Category.str.contains(cat)].

Is there some obvious reason while the cat variable is not being passed into the filter command?

Pardon the long-winded and lurid description but it has been frustrating that something seemingly straightforward will not perform.


Submit an answer

This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

Sign In or Sign Up to Answer

These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.

Want to learn more? Join the DigitalOcean Community!

Join our DigitalOcean community of over a million developers for free! Get help and share knowledge in Q&A, subscribe to topics of interest, and get courses and tools that will help you grow as a developer and scale your project or business.

Hi there,

At a first glance, it looks good. Can you share both the outgo and the ecats files here so that I could try and test this as well?

Regards, Bobby