posts - 280, comments - 318, trackbacks - 0

My Links

News

View Steve Michelotti's profile on LinkedIn

Twitter












Tag Cloud

Archives

Post Categories

Blend Bloggers

Bloggers that I follow

Books

F# Bloggers

F# Communities

F# Online Books

Fonts

HTML CSS ASP

Machine Learning

My Links

My Local UserGroups

My Online Presence

MY SA Links

Online Seminars

SA Software Companies

Web Design

F# – Depressingly simple text file access

 

Today I was doing some work work and kept having to parse a log file. After a few hours of doing the same routine stuff I thought I might try and automate it using F#.

It was depressingly simple to achieve…

Example Extract of Text File

*ERROR*   : Drawing num {^BASE^2 }783, surface num 1 uses invalid real colour number 2
_warning_ : The base file has 171 DLT types, the main file has 126.
_warning_ : AutoPlan error: There are no base corner units with autoplan 30 (see theme
*ERROR*   : Drawing num {^BASE^2 }790, surface num 1 uses invalid real colour number 5
_warning_ : The base file has 171 DLT types, the main file has 126.
_warning_ : AutoPlan error: There are no base corner units with autoplan 30 (see theme
*ERROR*   : Drawing num {^BASE^10 }600401, surface num 1, is an alias that depends on a DLT type with no entries
*ERROR*   : Drawing num {^BASE^10 }600405, surface num 1, is an alias that depends on a DLT type with no entries

So, what I wanted to achieve was to put all the Error entries together and then all the warning entries below it in a parse textfile..

The code looked something like this…

open System.IO
open System

let ParseFile = 
    let ParseFile = File.ReadAllLines("Z:\\HomeConcepts_07 Report.err")
    
    let ParseFileForErrors = 
        ParseFile
        |> Seq.filter(fun x -> if x.Contains("_warning_") then false else true)
    
    let ParseFileForWarnings = 
        ParseFile
        |> Seq.filter(fun x -> if x.Contains("_warning_") then true else false)

    let myFileSet = 
        let mySubFileSet = Seq.append ParseFileForErrors ParseFileForWarnings    
        let FileSummary = "Number of Entries : " + ParseFile.Length.ToString() + " Number of Errors : " + (ParseFileForErrors |> Seq.length).ToString()
        Seq.append ( FileSummary |> Seq.singleton) mySubFileSet
    
    File.WriteAllLines("Z:\\Test.txt", myFileSet)        

ParseFile
printfn "Done"

Explanation of Code

In ParseFile I create a sequence with each element as a line entry from the file.

I then filter the error elements, and generate a sequence, and do the same for the warning messages.

Finally in myFileSet I put all the sequences together and output them using the File.WriteAllLines

Print | posted on Tuesday, July 06, 2010 9:32 PM | Filed Under [ F# ]

Feedback

Gravatar

# re: F# – Depressingly simple text file access

This proved handy for working out the file access bit I needed for a Project Euler problem. I'm still learning how to use F# but as I'm getting used to recursion, tail recursion, mapping and so on I'm impressed with how elegant some things are. It's a shame the documentation is still playing catch-up compared to the likes of C#, but blogs like this provide us new F#-ers with invaluable hints.
7/7/2010 12:53 AM | David Burton
Gravatar

# re: F# – Depressingly simple text file access

Hey David...

Thanks for the comment... I'm in the same boat as you - new to F# - but loving it all the way.

7/7/2010 8:09 AM | MarkPearl
Gravatar

# re: F# – Depressingly simple text file access

I agree, the most depressing thing about the ease of writing file (and other) parsers in F# is the comparison with other methods. I ported one file parser for a commerical project I'm involved in and achieved something like a >75% reduction in code while simultaneously increasing the capabilities of the parser. It started as a proof-of-concept, but it looks like it's going to make it into production (after I finish a small adapter). I'm now porting two other file parsers for the same project.
7/7/2010 8:01 PM | TechNeilogy
Gravatar

# re: F# – Depressingly simple text file access

This could also be done quite easily in powershell if you just need a temporary script.
7/8/2010 6:59 AM | Kirb
Gravatar

# re: F# – Depressingly simple text file access

Originally I was going to give F# a brief look and then have a peek at other languages like Powershell, but I just got hooked up on F#.

1) reason why I like F# - it is so succint.
7/8/2010 8:16 AM | MarkPearl
Gravatar

# re: F# – Depressingly simple text file access

Nice work... You could also do a single pass by using List.partition or something like

let ParseFileForWarnings,ParseFileForErrors =
ParseFile
|> Seq.groupBy(fun x -> x.Contains("_warning_"))
|> Seq.toArray
|> (fun i -> snd i.[0],snd i.[1])

Also the sprintf function is nice, you can do:

let FileSummary = sprintf "Number of Entries : %d Number of Errors : %d" (ParseFileForErrors |> Seq.length) ParseFile.Length


12/24/2010 10:50 PM | wuxab
Post A Comment
Title:
Name:
Email:
Website:
Comment:
Verification:
 
 

Powered by: