Skip to content

Removed Read methods taking string parameters from TextLoader (#1797) #2323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 4, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions src/Microsoft.ML.Data/DataLoadSave/DataReaderExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Microsoft.Data.DataView;
using Microsoft.ML.Core.Data;
using Microsoft.ML.Data;

namespace Microsoft.ML
{
public static class DataReaderExtensions
{
public static IDataView Read(this IDataReader<IMultiStreamSource> reader, string path)
{
return reader.Read(new MultiFileSource(path));
}

public static IDataView Read(this IDataReader<IMultiStreamSource> reader, params string[] path)
{
return reader.Read(new MultiFileSource(path));
}
}
}
4 changes: 0 additions & 4 deletions src/Microsoft.ML.Data/DataLoadSave/Text/TextLoader.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1312,10 +1312,6 @@ public void Save(ModelSaveContext ctx)

public IDataView Read(IMultiStreamSource source) => new BoundLoader(this, source);

public IDataView Read(string path) => Read(new MultiFileSource(path));

public IDataView Read(params string[] path) => Read(new MultiFileSource(path));
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka Jan 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we prefer to have two methods because params is a bit slow one, and for performance we prefer to have one method for single file case (which is like 99%) and one for case with multiple file paths. #Closed


internal static TextLoader CreateTextReader<TInput>(IHostEnvironment host,
bool hasHeader = DefaultArguments.HasHeader,
char separator = DefaultArguments.Separator,
Expand Down
21 changes: 21 additions & 0 deletions src/Microsoft.ML.StaticPipe/LocalPathReader.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Microsoft.ML.Data;

namespace Microsoft.ML.StaticPipe
{
public static class LocalPathReader
{
public static DataView<TShape> Read<TShape>(this DataReader<IMultiStreamSource, TShape> reader, string path)
{
return reader.Read(new MultiFileSource(path));
}

public static DataView<TShape> Read<TShape>(this DataReader<IMultiStreamSource, TShape> reader, params string[] path)
{
return reader.Read(new MultiFileSource(path));
}
}
}
14 changes: 0 additions & 14 deletions src/Microsoft.ML.StaticPipe/TextLoaderStatic.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

using System;
using System.Collections.Generic;
using Microsoft.Data.DataView;
using Microsoft.ML.Core.Data;
using Microsoft.ML.Data;
using Microsoft.ML.StaticPipe.Runtime;
Expand Down Expand Up @@ -306,17 +305,4 @@ public TextLoader.Column Create()
}
}
}

public static class LocalPathReader
{
public static IDataView Read(this IDataReader<IMultiStreamSource> reader, params string[] path)
{
return reader.Read(new MultiFileSource(path));
}

public static DataView<TShape> Read<TShape>(this DataReader<IMultiStreamSource, TShape> reader, params string[] path)
{
return reader.Read(new MultiFileSource(path));
}
}
}