How to properly read j1939 messages from .asc file with cantools?

I'm trying to create a CAN logs converter from .asc files to .csv files (in human readable form). I'm somewhat successful. My code works fine with almost any database but j1939.dbc.

The thing is, that if I print out the messages read from the dbc file, I can see that the messages from j1939.dbc are read into the database. But it fails to find any of those messages in the processed log file. At the same time I can read the same file using Vector CANalyzer with no issues.

I wonder why this may happed and why it only affects the j1939.dbc and not the others.

I suspect that maybe the way I convert those messages is wrong because it never goes by the 'if msg_id in database:' line (and as mentioned above, those messages are certainly there because Vector CANalyzer works fine with them).

import pandas as pd
import can
import cantools
import time as t
from tqdm import tqdm
import re
import os
from binascii import unhexlify


dbcs = [filename.split('.')[0] for filename in os.listdir('./dbc/') if filename.endswith('.dbc')]
files = [filename.split('.')[0] for filename in os.listdir('./asc/') if filename.endswith('.asc')]
start = t.time() 

db = cantools.database.Database()

for dbc in dbcs:
    with open(f'./dbc/{dbc}.dbc', 'r') as f:
        db.add_dbc(f)


f_num = 1

for fname in files:
    print(f'[{f_num}/{len(files)}] Parsing data from file: {fname}')    
    log=can.ASCReader(f'./asc/{fname}.asc')
    entries = []
    all_msgs =[]


    message = {'Time [s]': ''}
    database = list(db._frame_id_to_message.keys())
    print(database)
    lines = sum(1 for line in open(f'./asc/{fname}.asc'))
    msgs = iter(log)

    try:
        for msg, i in zip(msgs, tqdm(range(lines))):

            msg = re.split("\\s+", str(msg))
            timestamp = round(float(msg[1]), 0)
            msg_id = int(msg[3], 16)

            try:
                data = unhexlify(''.join(msg[7:15]))
            except:
                continue

            if msg_id in database:
                if timestamp != message['Time [s]']:
                    entries.append(message.copy())
                    message.update({'Time [s]': timestamp})
                message.update(db.decode_message(msg_id, data))

    except ValueError:
        print('ValueError')
        
    df = pd.DataFrame(entries[1:])
    duration = t.time() - start
    df.to_csv(f'./csv/{fname}.csv', index=False)
    print(f'DONE IN {int(round(duration, 2)//60)}min{round(duration % 60, 2)}s!\n{len(df.columns)} signals extracted!')
    f_num += 1


Read more here: https://stackoverflow.com/questions/66385856/how-to-properly-read-j1939-messages-from-asc-file-with-cantools

Content Attribution

This content was originally published by Wojciech Wołujewicz at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: