# mdkir -p /share/cdh_repo # cd /share/cdh_repo
(Note: This can be any directory)
1) Download CDH parcel to the repo directory:
#pwd /share/cdh_repo #wget http://archive-primary.cloudera.com/cdh5/parcels/latest/CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel
2) Download the manifest.json file from the URL:
#pwd /share/cdh_repo #wget http://archive-primary.cloudera.com/cdh5/parcels/latest/manifest.json
3) Copy the hash code:
#pwd /share/cdh_repo
Open the manifest.json file from the URL:
http://archive-primary.cloudera.com/cdh5/parcels/latest/manifest.json
Find the section of the manifest that corresponds to the parcel you downloaded.
For example, if you are running CentOS 6 and copied the parcel file CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel, then you would look for the section:
"parcelName": "CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel",
"components": [
....
],
"replaces": "IMPALA, SOLR, SPARK",
"hash": "67fc4c86b260eeba15c339f1ec6be3b59b4ebe30"
},
Create an SHA file containing the hash code. For example:
# echo "67fc4c86b260eeba15c339f1ec6be3b59b4ebe30" > CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel.sha 4) Create a server location to be accessible by CM as a URL:
(You can use any other server. Ex: Apache)# cd /share # nohup python -m SimpleHTTPServer 9080 & # nohup: ignoring input and appending output to `nohup.out' Press Enter Now the folder "/share/cdh_repo" contains the below files/directories in it.
# ls -1 CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel.sha manifest.json This is our repo address now:
http://ip-address-of-the-machine:9080/cdh_repo/5) Go to Cloudera Manager
Administration > Settings > Parcels > Remote Parcel Repository URLs
Provide our server address, where we just created a repo:
http://ip-address-of-the-machine:9080/cdh_repo/Save and Click on 'Check for New Parcels'
You have your offline repo 🙂
Going one step ahead, now if we want other services like Spark to be able from the same URL.
1) Download the parcel file:#pwd /share/cdh_repo #wget http://archive-primary.cloudera.com/spark/parcels/latest/SPARK-0.9.0-1.cdh4.6.0.p0.98-el6.parcel 2) Create an SHA file containing the hash code. For example:
#pwd /share/cdh_repo #echo "dd9b61b3ef24b5c1e8ecee75f3123924796d660d" > SPARK-0.9.0-1.cdh4.6.0.p0.98-el6.parcel.sha 3) Now, in one directory there can be only one manifest.json file.
To add Spark, we have to append manifest details to the existing manifest.json we already have.#pwd /share/cdh_repo Open the manifest.json from the URL: http://archive-primary.cloudera.com/spark/parcels/latest/manifest.json
Search for the parcel name we just downloaded Ex: SPARK-0.9.0-1.cdh4.6.0.p0.98-el6.parcel
and copy the corresponding details. ( Highlighted text below is the newly added content)
{
"lastUpdated": 14054557850000,
"parcels": [
{
"parcelName": "CDH-5.1.0-1.cdh5.1.0.p0.53-precise.parcel",
"components": []
"replaces": "IMPALA, SOLR, SPARK",
"hash": "767cf5b995c38ea467270373dee09f471194b393"
}
,
{
"parcelName": "SPARK-0.9.0-1.cdh4.6.0.p0.98-el6.parcel",
"components": [
{ "name": "spark",
"version": "0.9.0-cdh4.6.0",
"pkg_version": "0.9.0",
"pkg_release": "1.cdh4.6.0.p0.46"
}
],
"depends": "CDH (>= 4.4), CDH (<< 5.0)",
"hash": "dd9b61b3ef24b5c1e8ecee75f3123924796d660d"
}
] }
4) Now we have CDH and Spark in our repo
#pwd /share/cdh_repo # ls -1 CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel CDH-5.1.0-1.cdh5.1.0.p0.53-el6.parcel.sha manifest.json SPARK-0.9.0-1.cdh4.6.0.p0.98-el6.parcel SPARK-0.9.0-1.cdh4.6.0.p0.98-el6.parcel.sha Comment below if you find this blog useful.
Hello Puneetha,
Thanks for the above info :-).
I want to create my own parcels with some script, tools ,jar… which needs to eb distributed across my cluster. could you please help me with this.
Nijanthan
+91-9962214433
I have gone through your blog followed all the steps but it didn’t come. Please help me. It ll be very great help.)
I have installed cloudera manager(5.4.7) and created Local parcel repositoty using SimpleHTTPServer with CDH-5.4.7-1.cdh5.4.7.p0.3-trusty.parcel, CDH-5.4.7-1.cdh5.4.7.p0.3-trusty.parcel.sha1 and manifest.json file and its accessible from brwoser but when I am putting this entry in Remote Parcel Repository URLs its not showing available parcels remotely. I have given permissions to local parcel like cloudera-scm:cloudera-scm. I have been through lots of website but I coudnt get a proper solution some people are saying use proxy setting in parcels and restart the server but still it didnt come for me please tell me what is the proper procedure for this getting done. Thanks in advance.