Sahara currently demands that the URL for an hdfs data source begins with the hdfs scheme "hdfs://". The UI enforces this restriction as well.
When the hdfs scheme is specified, hadoop requires the hostname and/or port to be specified as well. For use of an external hdfs, of course, the host/port are always necessary.
On a long running cluster, the local hdfs can be used by specifying the host/port of the local namenode in the path so that data can be read/written to the cluster hdfs, for example "hdfs://test-master:9000/user/hadoop/input". However, hadoop supports simpler absolute and relative paths to access the local hdfs without specification of the host and port.
For example (assuming the hadoop user), the relative path "output_path" will evaluate to
hdfs://namenode:port/user/hadoop/output_path
and the absolute path "/output_path" will evaluate to
hdfs://namenode:port/output_path
Sahara should relax the restriction that hdfs URLs start with "hdfs://", and allow absolute/relative paths to access the local hdfs.
The patch for this in the sahara api is simple. Do not require the hdfs scheme, but if a scheme is present enforce hdfs and hostname. The UI will need changes to allow paths without scheme (but the change can be tested with the CLI)
diff --git a/sahara/
index d2c8072..89d393c 100644
--- a/sahara/
+++ b/sahara/
@@ -69,8 +69,9 @@ def _check_
if len(data['url']) == 0:
raise ex.InvalidExcep
url = urlparse.
- if url.scheme != "hdfs":
- raise ex.InvalidExcep
- if not url.hostname:
- raise ex.InvalidExcep
- "cannot determine a hostname")
+ if url.scheme:
+ if url.scheme != "hdfs":
+ raise ex.InvalidExcep
+ if not url.hostname:
+ raise ex.InvalidExcep
+ "cannot determine a hostname")