Geeks With Blogs
Mark Pearl

 

Today I was doing some work work and kept having to parse a log file. After a few hours of doing the same routine stuff I thought I might try and automate it using F#.

It was depressingly simple to achieve…

Example Extract of Text File

*ERROR*   : Drawing num {^BASE^2 }783, surface num 1 uses invalid real colour number 2
_warning_ : The base file has 171 DLT types, the main file has 126.
_warning_ : AutoPlan error: There are no base corner units with autoplan 30 (see theme
*ERROR*   : Drawing num {^BASE^2 }790, surface num 1 uses invalid real colour number 5
_warning_ : The base file has 171 DLT types, the main file has 126.
_warning_ : AutoPlan error: There are no base corner units with autoplan 30 (see theme
*ERROR*   : Drawing num {^BASE^10 }600401, surface num 1, is an alias that depends on a DLT type with no entries
*ERROR*   : Drawing num {^BASE^10 }600405, surface num 1, is an alias that depends on a DLT type with no entries

So, what I wanted to achieve was to put all the Error entries together and then all the warning entries below it in a parse textfile..

The code looked something like this…

open System.IO
open System

let ParseFile = 
    let ParseFile = File.ReadAllLines("Z:\\HomeConcepts_07 Report.err")
    
    let ParseFileForErrors = 
        ParseFile
        |> Seq.filter(fun x -> if x.Contains("_warning_") then false else true)
    
    let ParseFileForWarnings = 
        ParseFile
        |> Seq.filter(fun x -> if x.Contains("_warning_") then true else false)

    let myFileSet = 
        let mySubFileSet = Seq.append ParseFileForErrors ParseFileForWarnings    
        let FileSummary = "Number of Entries : " + ParseFile.Length.ToString() + " Number of Errors : " + (ParseFileForErrors |> Seq.length).ToString()
        Seq.append ( FileSummary |> Seq.singleton) mySubFileSet
    
    File.WriteAllLines("Z:\\Test.txt", myFileSet)        

ParseFile
printfn "Done"

Explanation of Code

In ParseFile I create a sequence with each element as a line entry from the file.

I then filter the error elements, and generate a sequence, and do the same for the warning messages.

Finally in myFileSet I put all the sequences together and output them using the File.WriteAllLines

Posted on Tuesday, July 6, 2010 9:32 PM F# | Back to top


Comments on this post: F# – Depressingly simple text file access

# re: F# – Depressingly simple text file access
Requesting Gravatar...
This proved handy for working out the file access bit I needed for a Project Euler problem. I'm still learning how to use F# but as I'm getting used to recursion, tail recursion, mapping and so on I'm impressed with how elegant some things are. It's a shame the documentation is still playing catch-up compared to the likes of C#, but blogs like this provide us new F#-ers with invaluable hints.
Left by David Burton on Jul 07, 2010 12:53 AM

# re: F# – Depressingly simple text file access
Requesting Gravatar...
Hey David...

Thanks for the comment... I'm in the same boat as you - new to F# - but loving it all the way.

Left by MarkPearl on Jul 07, 2010 8:09 AM

# re: F# – Depressingly simple text file access
Requesting Gravatar...
I agree, the most depressing thing about the ease of writing file (and other) parsers in F# is the comparison with other methods. I ported one file parser for a commerical project I'm involved in and achieved something like a >75% reduction in code while simultaneously increasing the capabilities of the parser. It started as a proof-of-concept, but it looks like it's going to make it into production (after I finish a small adapter). I'm now porting two other file parsers for the same project.
Left by TechNeilogy on Jul 07, 2010 8:01 PM

# re: F# – Depressingly simple text file access
Requesting Gravatar...
This could also be done quite easily in powershell if you just need a temporary script.
Left by Kirb on Jul 08, 2010 6:59 AM

# re: F# – Depressingly simple text file access
Requesting Gravatar...
Originally I was going to give F# a brief look and then have a peek at other languages like Powershell, but I just got hooked up on F#.

1) reason why I like F# - it is so succint.
Left by MarkPearl on Jul 08, 2010 8:16 AM

# re: F# – Depressingly simple text file access
Requesting Gravatar...
Nice work... You could also do a single pass by using List.partition or something like

let ParseFileForWarnings,ParseFileForErrors =
ParseFile
|> Seq.groupBy(fun x -> x.Contains("_warning_"))
|> Seq.toArray
|> (fun i -> snd i.[0],snd i.[1])

Also the sprintf function is nice, you can do:

let FileSummary = sprintf "Number of Entries : %d Number of Errors : %d" (ParseFileForErrors |> Seq.length) ParseFile.Length


Left by wuxab on Dec 24, 2010 10:50 PM

Your comment:
 (will show your gravatar)


Copyright © MarkPearl | Powered by: GeeksWithBlogs.net