在scala中处理csv

我正在使用scala 2.7.7,并希望解析CSV文件并将数据存储在SQLite数据库中.

我最终使用OpenCSV java库来解析CSV文件,并使用sqlitejdbc库.

使用这些java库使我的scala代码看起来几乎与Java代码相同(没有分号和val/var)

由于我正在处理java对象,我不能使用scala list,map等,除非我做scala2java转换或升级到scala 2.8

有没有办法可以使用我不知道的scala位进一步简化我的代码？

val filename = "file.csv";
val reader = new CSVReader(new FileReader(filename))
var aLine = new Array[String](10)
var lastSymbol = ""
while( (aLine = reader.readNext()) != null ) {
    if( aLine != null ) {
        val symbol = aLine(0)
        if( !symbol.equals(lastSymbol)) { 
            try {
                val rs = stat.executeQuery("select name from sqlite_master where name='" + symbol + "';" )
                if( !rs.next() ) {
                    stat.executeUpdate("drop table if exists '" + symbol + "';")
                    stat.executeUpdate("create table '" + symbol + "' (symbol,data,open,high,low,close,vol);")
                }
            }
            catch {
              case sqle : java.sql.SQLException =>
                 println(sqle)

            }
            lastSymbol = symbol
        }
        val prep = conn.prepareStatement("insert into '" + symbol + "' values (?,?,?,?,?,?,?);")
        prep.setString(1, aLine(0)) //symbol
        prep.setString(2, aLine(1)) //date
        prep.setString(3, aLine(2)) //open
        prep.setString(4, aLine(3)) //high
        prep.setString(5, aLine(4)) //low
        prep.setString(6, aLine(5)) //close
        prep.setString(7, aLine(6)) //vol
        prep.addBatch()
        prep.executeBatch()
     }
}
conn.close()

Arjan Blokzi.. 10

如果您有一个简单的CSV文件,则可以选择根本不使用任何CSV库,而只需在Scala中解析它,例如:

case class Stock(line: String) {
  val data = line.split(",")
  val date = data(0)
  val open = data(1).toDouble
  val high = data(2).toDouble
  val low = data(3).toDouble
  val close = data(4).toDouble
  val volume = data(5).toDouble
  val adjClose = data(6).toDouble

  def price: Double = low
}

scala> import scala.io._

scala> Source.fromFile("stock.csv") getLines() map (l => Stock(l))
res0: Iterator[Stock] = non-empty iterator


scala> res0.toSeq  
res1: Seq[Stock] = List(Stock(2010-03-15,37.90,38.04,37.42,37.64,941500,37.64), Stock(2010-03-12,38.00,38.08,37.66,37.89,834800,37.89) //etc...

这样可以使用完整的Scala集合API.

如果您更喜欢使用解析器组合器,那么github上还有一个csv解析器组合器的示例.

1> Arjan Blokzi..：

---

如果您有一个简单的CSV文件,则可以选择根本不使用任何CSV库,而只需在Scala中解析它,例如:

case class Stock(line: String) {
  val data = line.split(",")
  val date = data(0)
  val open = data(1).toDouble
  val high = data(2).toDouble
  val low = data(3).toDouble
  val close = data(4).toDouble
  val volume = data(5).toDouble
  val adjClose = data(6).toDouble

  def price: Double = low
}

scala> import scala.io._

scala> Source.fromFile("stock.csv") getLines() map (l => Stock(l))
res0: Iterator[Stock] = non-empty iterator


scala> res0.toSeq  
res1: Seq[Stock] = List(Stock(2010-03-15,37.90,38.04,37.42,37.64,941500,37.64), Stock(2010-03-12,38.00,38.08,37.66,37.89,834800,37.89) //etc...

这样可以使用完整的Scala集合API.

如果您更喜欢使用解析器组合器,那么github上还有一个csv解析器组合器的示例.

---

如果任何字段中包含逗号,则分割为逗号将不起作用,例如"San Jose,California","SomeOtherData"

---

@rancidfishbreath CSV RFC不同意你的观点:https://tools.ietf.org/html/rfc4180.只要它们被正确转义,在CSV中使用非分隔逗号是完全有效的.非分离换行符也是如此.否则格式将受到严重限制.