Stories

APE Tags with taglib-sharp

Posted on Updated on

I’ve been working on a solution recently to try and read and write data tags from an APE data set in an MP3.

This usually works, however, I’ve discovered a problem with the library I’ve been using when it’s returning incorrect data, of which has been confirmed by a HEX editor.

I’ve been working with a tagging  format which has various non-standard fields are saved by myself, but also one done with a third party audio program, which reads and writes the tag of “XRestrictions”, and where expected the subsequent next bytes represent data values – the data of the tag.

This has been rather interesting, firstly, because the program writing the initial tags is  written Delphi, but mainly because of how it is handling the data it’s writing.  It’s storing a sequence of string hex values, that represents dates and times, but the HEX date values are interestingly stored around the wrong way.

Let me explain further.

The is the HEX code from the file, as per a HEX editor.

Screenshot 2019-04-24 23.37.41

The 40AA value is a date value.

But, what the Delphi code has done, is invert the HEX date value.  In short, the dates are calculating the days from 31 December 1899.

If we invert those values, it is AA40, and we convert from HEX to Decimal, it’s actually 43584. Storing values like this is very efficient, but the values are swapped – I have no idea why though.

Either way, we have the following values now.

Start Date: AA400000
End Date: AA410000

You can see the workings out here for calculating how many days since 31 Dec 1899, which is 43584 and brings us to: Tuesday, 30 April 2019.

So, moving forward I can now calculate these given values. But, how did I actually get the HEX values, and what went wrong in the first place with the library?

I’ve been using the TagLib-Sharp library for VB.NET with no problems. However, once I started reading these date tags, that is where it all went wrong.

Plain ASCII data is stored nicely… the standard characters…  But, once it gets to other values, the library kind of freaks out and gives the wrong response.

Earlyier we saw what the correct HEX values should have beeen. As shown below,  these are the results of what the library gave out, and thus, you will see why I discovered that I needed to wrint my own code to extract the correct data, and how I did it – in very rough un-edited code.

This was the raw ASCII text of the “XRestrictions” APE tag  given by the  TagLib-Sharp library:

@?, , A?, , , , , , , , Q, , , , , Q, , , , , Q, , , , , Q, , , , , Q, , , , , Q, , , , , Q

This can be extracted with the following:

Dim XRestriction_value As String = GetField(“XRestrictions”, Full_Path)
Console.WriteLine(“XRestriction_value: ” + XRestriction_value)

Unfortunately it was wrong, and it took me much of the day to establish what was wrong.

Next, I tried converting the raw value to HEX values, which is also incorrect via TagLib-Sharp:

403F0000413F000000001000000007F511001000000007F511001000000007F511001000000007F511001000000007F511001000000007F511001000000007F511

This can be extracted with the following:

Dim test As String = StringToHex(GetField_Date(“XRestrictions”, Full_Path))
test = test.Replace(“2C”, “”) ‘ remove commas
test = test.Replace(“20”, “00”) ‘ remove spaces
Console.WriteLine(“raw: ” + test)

This was also incorrect for some reason. Notice the text replacements? This created a very long string with more unnecessary characters as well.

After a day of trying to figure this all out, here is the correct HEX via my search code (with a bit of help from others).

40AA0041AA0000100007F5110100007F5110100007F5110100007F5110100007F5110100007F5110100007F511010000000

Here is my spaghetti code extracting the correct values.

So, the way I did it in the end, was to do a search in the file contents looking for the phrase “XRestrictions”. Once I got the offset, I could then read in the data in bytes at (14 characters + the offset) to get the correct starting point, and then read the data.

Debug.Print(Full_Path)
Dim XRestriction_value As String = GetField("XRestrictions", Full_Path)
XRestriction.Text = XRestriction_value

Console.WriteLine("XRestriction_value: " + XRestriction_value)

Dim test As String = StringToHex(GetField_Date("XRestrictions", Full_Path))

test = test.Replace("2C", "") ' remove commas
test = test.Replace("20", "00") ' remove spaces

Console.WriteLine("raw: " + test)

SPLDateRangeraw.Text = test


' works
Dim SearchOffset As Integer

Dim b2() As Byte = IO.File.ReadAllBytes(Full_Path)

Dim encoding As New System.Text.ASCIIEncoding
Dim SearchString As String = "XRestrictions"
Dim bSearch As Byte() = encoding.GetBytes(SearchString)
Dim bFound As Boolean = True

For i2 = 0 To b2.Length - bSearch.Length - 1
If b2(i2) = bSearch(0) Then
bFound = True
For j As Integer = 0 To bSearch.Length - 1
If b2(i2 + j) <> bSearch(j) Then
bFound = False
Exit For
End If
Next
If bFound Then
'MsgBox(SearchString & " found at byte offset: " & i)
Console.WriteLine(" found at byte offset: " & i2)
SearchOffset = i2
End If
End If
Next

' end works.

Dim helloworld As String

' Get the file name.
Dim file_name As String = Full_Path
' Open the file.

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(file_name, FileMode.Open, FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
Console.WriteLine("lBytes 2: " + lBytes.ToString)
oBinaryReader.BaseStream.Position = SearchOffset + 14 ' for the word search offset.
Dim fdi As Integer = 0
For i = 0 To 100
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()
helloworld = ""
For loop1 = 0 To 80
'helloworld = helloworld & fileData(loop1) & "/"

helloworld = helloworld & Convert.ToString(Convert.ToInt32(fileData(loop1)), 16).ToUpper
Next loop1

Console.WriteLine("Correct Values: " + helloworld)


Dim StartOutput As String
StartOutput = test.Substring(test.Length - 4, 4)
Dim Startoutputleft As String = helloworld.Substring(0, 2)
Dim Startoutputright As String = helloworld.Substring(2, 2)
Dim Startnewoutput = Startoutputright + Startoutputleft
Dim FinalHEXStartDate = Startoutputright + Startoutputleft + "00" + "00"
Console.WriteLine("Start Date newoutput: " + FinalHEXStartDate)


Dim EndStartOutput As String
EndStartOutput = test.Substring(test.Length - 4, 4)
Dim EndStartoutputleft As String = helloworld.Substring(6, 2)
Dim EndStartoutputright As String = helloworld.Substring(8, 2)
Dim EndStartnewoutput = EndStartoutputright + EndStartoutputleft
Dim EndFinalHEXStartDate = EndStartoutputright + EndStartoutputleft + "00" + "00"
Console.WriteLine("End Date newoutput: " + EndFinalHEXStartDate)

It’s a crazy thing to have to figure out, but if you are working on VB.NET, then this may help a little bit with your MP3 APE metadata.

Either way, this is the full output – set for April 30, 2019.

C:\Users\steve\Desktop\test 2.mp3
XRestriction_value: @?, , A?, , , , , , , , Q, , , , , Q, , , , , Q, , , , , Q, , , , , Q, , , , , Q, , , , , Q
raw: 403F0000413F000000001000000007F511001000000007F511001000000007F511001000000007F511001000000007F511001000000007F511001000000007F511
found at byte offset: 279628

APE data fields are great and are totally customisable which don’t have to adhere to  ID3 name field standards. This is especially helpful when you want to embed application specific tags in files without effecting the usability of the file.

Next step, getting the data written correctly. This shouldn’t be too hard in theory.

Advertisements

Fusio

Posted on Updated on

After playing with the Fusio project for a while, and trying it out in probably the most unorthodox way, and trying to read between the lines with the documentation, I’ve managed to get a list of data from the php_sandbox.

For example, this is the php code:

<?php
$connection = $connector->getConnection('System');

$id = (int)$request->getParameters('user_id');
$another_string =  $request->getParameters('another_string');
$todo = $connection->fetchAll('Select *, date as datecol from fusio_app where user_id = ' . $id . ' order by datecol ASC');
$count  = $connection->fetchColumn('SELECT COUNT(*) FROM fusio_app where user_id = ' . $id );

return $response->build(200, [], 
[
    'another_string' => $another_string,
    'totalResults' => $count,
    'results' => $todo
    ]);

URI Fragments:

user_id=1&another_string=Steve

Method:

GET

 

HTTP example GET:
 https://domain.com/fusio/public/index.php/get_test?user_id=1

And, that should then give you a response of the “apps” currently listed in the main database, if this is a fresh install.

Response:

{
    "another_string": {
        "user_id": "1",
        "another_string": "Steve"
    },
    "totalResults": "3",
    "results": [
        {
            "id": "3",
            "user_id": "1",
            "status": "1",
            "name": "YSG",
            "url": "",
            "parameters": null,
            "app_key": "xxxxxxxxxxxxxxxxxx",
            "app_secret": "xxxxxxxxxxxxxxxxxx",
            "date": "2019-03-24 14:22:36",
            "datecol": "2019-03-24 14:22:36"
        },
        {
            "id": "1",
            "user_id": "1",
            "status": "1",
            "name": "Backend",
            "url": "http://fusio-project.org",
            "parameters": "",
            "app_key": "xxxxxxxxxxxxxxxxxx",
            "app_secret": "xxxxxxxxxxxxxxxxxx",
            "date": "2019-03-24 22:08:13",
            "datecol": "2019-03-24 22:08:13"
        },
        {
            "id": "2",
            "user_id": "1",
            "status": "1",
            "name": "Consumer",
            "url": "http://fusio-project.org",
            "parameters": "",
            "app_key": "xxxxxxxxxxxxxxxxxx",
            "app_secret": "xxxxxxxxxxxxxxxxxx",
            "date": "2019-03-24 22:08:13",
            "datecol": "2019-03-24 22:08:13"
        }
    ]
}

Of course this is just a test, and should be accompanied with better security policies and the like, with a token response or similar, but it’s a general quick request.

There is so much more room for better documentation on the site, as most what is currently written, is that it assumed you know the majority of it already.

I’ll post more and update more as I continue to discover things.

This documentation has a few examples:
https://www.fusio-project.org/documentation/recipes/select-rows

Getting SMS messages from the Huawei E3372 LTE modem

Posted on Updated on

If you’ve bought a modem from Optus, here in Australia, you may be delighted – and disappointed that the SMSs that are received on the Modem, are now served through a web interface.

I wasn’t happy about this, as I wanted to interrogate the SMS database through the “Mobile Partner” program. So, because Optus have custom firmware it’s basically not possible to do this (Yes, I was on the phone to both parties), it has caused me to look elsewhere. If you wanted to use a custom firmware, go for it, but I couldn’t be bothered.

Specifically, I have an E3372 modem.

I came across this post and information, it got me thinking…

The same hackaround can be used to programatically send SMS from the LAN using the E3372 API, here’s a simple script that does just that:

#!/bin/bash

DATA=`curl http://192.168.8.1/api/webserver/SesTokInfo`
SESSION_ID=`echo "$DATA" | grep "SessionID=" | cut -b 10-147`
TOKEN=`echo "$DATA" | grep "TokInfo" | cut -b 10-41`

curl http://192.168.8.1/api/sms/send-sms -H "Cookie: $SESSION_ID" -H "__RequestVerificationToken: $TOKEN" --data "<?xml version='1.0' encoding='UTF-8'?><request><Index>-1</Index><Phones><Phone>$1</Phone></Phones><Sca></Sca><Content>$2</Content><Length>-1</Length><Reserved>1</Reserved><Date>-1</Date></request>"

Use it like this:

./send_sms.sh +1234567890 "Hello world!"

 

After studying the code a little, I’ve been able to discover that the SMS messages can easily be read from the stick, as XML.

Request URL:http://192.168.8.1/api/sms/sms-list
Request method:POST
Remote address:192.168.8.1:80
What it should do, is return an response of XML eg:
<?xml version="1.0" encoding="utf-8"?>
<response>
<Count>12</Count>
<Messages>
<Message>
<Smstat>1</Smstat>
<Index>40013</Index>
<Phone>OPTUS</Phone>
<Content>An SMS you tried to send or receive has failed as you are out of credit for that message type. Recharge now to allow future SMS messages to be sent or received.</Content>
<Date>2019-02-13 16:04:02</Date>
<Sca></Sca>
<SaveType>4</SaveType>
<Priority>0</Priority>
<SmsType>1</SmsType>
</Message>
<Message>
<Smstat>0</Smstat>
<Index>40011</Index>
<Phone>OPTUS</Phone>
<Content>An SMS you tried to send or receive has failed as you are out of credit for that message type. Recharge now to allow future SMS messages to be sent or received.</Content>
<Date>2019-02-13 16:03:20</Date>
<Sca></Sca>
<SaveType>4</SaveType>
<Priority>0</Priority>
<SmsType>1</SmsType>
</Message>
</Messages>
</Response>

 

This should be enough to get me started, and once again (possibly) start to read SMS messages from my (new) USB Modem.

While I haven’t tried this method fully yet, but in theory this will be completely possible.

This may also remove the need for Gammu as well.

For other valuable information on these devices (which are all very similar)
https://blog.hqcodeshop.fi/archives/259-Huawei-E5186-AJAX-API.html
http://www.gnuton.org/blog/2015/07/huawei-e3372/
http://blog.asiantuntijakaveri.fi/2015/07/convert-huawei-e3372h-153-from.html
https://sh.com.hr/en/modificiranje-huawei-e3372-lte-sticka/

Show a single WordPress article post in an Ionic Angular app.

Posted on Updated on

It took me a little while to do this nicely, but I’m happy with this version:    This will help you get a specific WP post with a wp_id.

This is not a full example, but close enough. If you know what you are doing, this should be easy for you.

 

 

I’m calling this code from another page which will link to the WP article.

<a style="width:100%"style="text-decoration:none;"class="item"href="#"(click)="OpenSinglePost(6872, 'Custom Article title ')">Show the Article</a>
The code behind on the .ts file:
OpenSinglePost(postid, pagetitle: string)
{
  this.navCtrl.push(WpArticleItemPage, {postId: postid, pagetitle: pagetitle });
}

Showing the actual WP article page:

 

 

wp-article-item.ts

 

import { Component } from '@angular/core';
import { IonicPage, NavController, NavParams, LoadingController, Loading } from 'ionic-angular';
import { ProgressHttp } from "angular-progress-http";

@IonicPage()
@Component({
selector: 'page-wp-article-item',
templateUrl: 'wp-article-item.html',
})

export class WpArticleItemPage
{
postId: number;
loader: Loading;
post_title: string;
post_content: string;
url: string;

constructor(
public navCtrl: NavController,
public navParams: NavParams,
public loadingCtrl: LoadingController,
private http: ProgressHttp)
{

this.presentLoading();
this.postId = navParams.get('postId');
this.post_title = navParams.get('pagetitle');
console.log("this.postId: " + this.postId);
this.url ='https://www.c4israel.com.au/wp-json/wp/v2/pages/'+this.postId;
this.http
.get( this.url ) // The URL
.map( res => res.json()) // putting an async makes it get stuck forever.
.subscribe( data =>
{

//this.post_title = data.title.rendered;
this.post_content =this.striplinks(data.content.rendered);
console.log(data);
this.loader.dismiss()
})
};

// This currently solves the URL problem

striplinks(text)
{
var re = /<a\s.*?href=[\"\'](.*?)[\"\']*?>(.*?)<\/a>/g;
var str = text;
var subst ='$2';
var result = str.replace(re, subst);
return result;
}

presentLoading() {
this.loader =this.loadingCtrl.create({
content: "Retrieving data..."
});

this.loader.present();
}
}

Yes, WP doesn’t format code nicely – I know.

Now for the front-end and display the WP article.

 

wp-article-item.html

<ion-header>
<ion-navbar color="primary">
<ion-title>{{ post_title }}</ion-title>
</ion-navbar>
</ion-header>

<ion-content padding>
< d i v [innerHTML]="post_content"></ d i v >
</ion-content>

(Just fix the div tags up. It's a WordPress thing)

Teamviewer not reloading after I quit it

Posted on

This has been an ongoing problem for a long time now. If I was to ever quit Teamviewer, it would cease to reopen, and instead, just give me a prompt saying “TeamViewer is trying to install a new helper tool”.

Today, I finally discovered the answer to my troubles, and hopefully a solution that will work.

I discovered it here.

What happening on my computer was that the launch daemon for the helper tool was not properly started.

So, to fix the problem, I ran this from the Mac terminal:

sudo launchctl load -w /Library/LaunchDaemons/com.teamviewer.Helper.plist

Teamviewer then finally loaded. I haven’t seen anyone else with this issue, so I don’t know why it is doing it, but, until the Teamviewer provide a proper solution, this work around will have to do.

Sadly, even with the latest version (14), this is still current.

Reading text Data from a USB Serial Device in Python on a raspberry pi.

Posted on Updated on

Programming in python is very new to me, and I’m only doing this out of necessity, as it appears it’s the most easy and common method on the raspberry pi. So, here goes…

This code reads data from a USB device which issues text on a serial port, where I can then disseminate the data.

And yes, this is initial version which hasn’t been tested 100% yet.

 

import serial
import time
import csv

port = '/dev/serial/by-id/usb-Prolific_Technology_Inc._USB-Serial_Controller_D-if00-port0'
#port = '/dev/ttyUSB0'
baud = 9600

ser = serial.Serial(port, baud, timeout=1)
ser.flushInput()

print(ser.name)

oldline = []

while True:
    try:
        #ser_bytes = ser.readline()
        #decoded_bytes = float(ser_bytes[0:len(ser_bytes)-2].decode("utf-8"))
        #print(decoded_bytes)
        
        
        line = ser.readline()                 # read bytes until line-ending
        line = line.decode(encoding='UTF-8')  # convert to string
        #line = line.rstrip('\r\n')            # remove line-ending characters
        
        split_line = line.splitlines()
        
        if oldline != split_line:      
            with open("test_data.csv","a") as f:
                for item in split_line:
                    writer = csv.writer(f,delimiter=",")
                    writer.writerow([time.time(), item])
                
        oldline = split_line
        
    except:
        print("exiting")
        #print("Keyboard Interrupt")
        break

I have now added it here: https://github.com/vrdriver/Serial-PI-thon

Retrieving Backups with Raspberry Pi and sftp

Posted on Updated on

Following on from my recent MYSQL DB, it’s one thing to back up the data, but you have to get the data off the server.

So, with my trusty Raspberry pi, that is going to use considerable less power than my NAS (which has been used for VMs to do this), it’s now time to get some scripts working on the pi to do the offline/off server backups.

 

The biggest problem is finding a solution to download the data securely. It’s easy to use plain FTP, which is not secure, but I need to have SFTP.

So, as it turns out, there is a program called sftp built in to the pi.

I went down the path of using sshpass. My FTP password has to be saved somewhere, and as it’s only for SQL backups, I’m not too concerned.

 

You have a few options other than using public key authentication. I have take this example from here.

  1. Use sshpass (less secured but probably that meets your requirement)

If you decide to give sshpass a chance here is a working script snippet to do so:

You should copy all of this in to a script.sh file.

export SSHPASS=your-password-here
sshpass -e sftp -P PORTNUMBER -oBatchMode=no -b - sftp-user@remote-host << !
   
   lcd local_backup_path1
   cd db_backup_path1 
   get *.gz
   cd..
   lcd ..

   lcd local_backup_path2
   cd db_backup_path2 
   get *.gz
   cd ..
   lcd ..

   bye
!

Please note the PORTNUMBER there. It has also be a capital for it to work.

Then, run the script with sh ./script.sh