Help me clean my Python code

[DrJohnZoidberg]

There is a little method in an app I'm writing that processes GET parameters sent from a web request for the Jquery plugin Datatables (https://www.datatables.net/).

The parameters it sends through look like this:

columns[0][data]:id
columns[0][name]:
columns[0][searchable]:true
columns[0][orderable]:true
columns[0][search][value]:
columns[0][search][regex]:false
columns[1][data]:date
columns[1][name]:
columns[1][searchable]:true
columns[1][orderable]:true
columns[1][search][value]:
columns[1][search][regex]:false

etc..

I need to extract these into something Python can understand. If these parameters were not arrays it would be easy (read, I would understand how to do it).

What I've written now works but it relies on the list being sorted correctly and statements being in the correct order:

def process_kwargs(kwargs):

        column_parameters = []

        for kwarg in sorted(kwargs):
            if re.search(r"\[(\w+)\]", kwarg) and kwarg[:7] == 'columns':
                parameters = re.findall(r"\[(\w+)\]", kwarg)
                array_index = ''
                for parameter in parameters:
                    pass_complete = False
                    if parameter.isdigit():
                        array_index = parameter
                    if parameter == 'data':
                        data = kwargs.get('columns[' + array_index + '][data]', "")
                    if parameter == 'orderable':
                        orderable = kwargs.get('columns[' + array_index + '][orderable]', "")
                    if parameter == 'searchable':
                        searchable = kwargs.get('columns[' + array_index + '][searchable]', "")
                        pass_complete = True
                    if pass_complete:
                        row = {'index': int(array_index),
                               'data': data,
                               'searchable': searchable,
                               'orderable': orderable}
                        column_parameters.append(row)

        return sorted(column_parameters, key=lambda i: i['index'])

Here I'm extracting the fields for data, orderable and searchable, but I have specifically check for each one.

How do I write this so it iterates through all of the parameters and extracts them without me telling it which ones to extract? I'm pretty sure this is such a simple thing but I cannot figure it out.

My Python-foo is weak, assistance I need.

TIA

 

[letslogic.com]

Why not use json?

 

[DrJohnZoidberg]

Why not use json?

Could you elaborate? Use JSON for what?

 

[Rickster]

 

[letslogic.com]

Access your data easily without if's and regex.

 

[DrJohnZoidberg]

Access your data easily without if's and regex.

You're being very vague. What must be JSON? Are you referring to the received GET parameters? I cannot change these, this is how the data comes from Datatables. Well, I probably could change it but then it won't work with any other non-hacked version of Datatables. Here is the reference page for the Datatables parameters: https://www.datatables.net/manual/server-side

 

[letslogic.com]

Basically, you can use json to parse your data and access it by name...

https://docs.python.org/3/library/json.html

 

[Necropolis]

Basically, you can use json to parse your data and access it by name...

https://docs.python.org/3/library/json.html

He would have to convert the incoming params to JSON first and then parse them - seems like a giant waste of time.

 

[DrJohnZoidberg]

I've left it as is for now. It's working (had a little bug earlier where it bugged out if there were 10 or more params, but fixed that).

I'll come back to this when part when I've finished the rest of it.

 

[letslogic.com]

He would have to convert the incoming params to JSON first and then parse them - seems like a giant waste of time.

Not entirely sure what you mean but if you're concerned about converting arrays to json, it happens so fast you wouldn't even notice it.

 

[DrJohnZoidberg]

Not entirely sure what you mean but if you're concerned about converting arrays to json, it happens so fast you wouldn't even notice it.

But how would I convert it to JSON and how would it be more efficient than what I'm already doing?

Remember my strings looks like this: columns[0][search][value]="test1", columns[0][search][regex]="", columns[1][search][value]="test2", columns[1][search][regex]=""

How do I get from that to this:

{
columns: [
  {"search": {
    "value": "test",
    "regex": ""
    }
  },
  {"search": {
    "value": "test2",
    "regex": ""
    }
  }]
}

 

[[)roi(]]

https://datatables.net/reference/api/ajax.json()

 

[DrJohnZoidberg]

https://datatables.net/reference/api/ajax.json()

Thanks, but that's for data that has already been requested. It doesn't send the parameters for "building" the table to the server side script.

 

[[)roi(]]

Thanks, but that's for data that has already been requested. It doesn't send the parameters for "building" the table to the server side script.

Sure, wasn't clear.
This could help get the output into a json format;
Basically you need to config datatables to convert the POST to json before sending it.

The datatables initialization to send JSON looks something like this:

ajax: {
  url: 'myUrl',
  data: function ( d ) {
    return { 'd': JSON.stringify( d ) };
  }
}

This github issue deals with this:
:https://github.com/DataTables/DataTables/issues/311

 

[Beachless]

There is quite a bit you can do but at least make use of a switch type statement (elif in python) instead of all the ifs on "parameter".

 

[DrJohnZoidberg]

Sure, wasn't clear.
This could help get the output into a json format;
Basically you need to config datatables to convert the POST to json before sending it.

The datatables initialization to send JSON looks something like this:

ajax: {
  url: 'myUrl',
  data: function ( d ) {
    return { 'd': JSON.stringify( d ) };
  }
}

This github issue deals with this:
:https://github.com/DataTables/DataTables/issues/311

Aha! This is what I'm looking for, thanks dude.

I'd still have to keep my current method for the regular HTTP requests (for compatibility) but having the JSON option makes it so much easier to process those parameters in Python.

Awesome find.

 

[[)roi(]]

Good to hear. Python-foo back on track.

 

[retromodcoza]

I haven't done python.

However , the PHP explode functions Python equivalent is string.split(the_string, the_separator)

since these are strings , split by [

so the whole thing would be something like (heavy pseudo code here)
foreach (string in strings) {

array = string.split(the_string, '[,:');

array[1] = array[1] .replace("]", "");
array[2] = array[2] .replace("]", "");

if len(array ==4) {
array[3] = array[3].replace(":", "");
array[4]= array[3];
}

if len(array ==5) {
array[3] = array[3].replace("]", "");
array[4] = array[4].replace(":", "");
}

my_list.extend([array])

}

Now your list has arrays with array[2] as they key , array[3] as any additional attributes and array[4] with the value which should be easy to call

Note : this code will not just work. You need to interpret it.

 

[DrJohnZoidberg]

Finally got to re-writing this class today. Ended up discarding a lot of the old stuff and now just accept a serialised json string for the parameters - much easier.

If anyone is looking for a Python Datatables connector for sqlite, then you can take a look here: https://github.com/drzoidberg33/plexpy/blob/master/plexpy/datatables.py

There are a few dependancies from my project in it but would be really easy to remove them and replace.

Supports the following functionality:

Pagination - yes
Filtering - yes
Sorting - yes
Multi-column sorting - yes
Individual column filter - no
Column filter enabled check (searchable_) - yes
Column sort enabled check (orderable) - yes
Regex support for filtering - no